SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity
Files
3104036e7f1a3cd6e07a69d648c3597de32f72fe
HuggingFace_transformer/tests
History
Manuel R. Ciosici 3104036e7f Add support for bitsandbytes (#15622) 
* Add initial BNB integration

* fixup! Add initial BNB integration

* Add bnb test decorator

* Update Adamw8bit option name

* Use the full bnb package name

* Overide bnb for all embedding layers

* Fix package name

* Formatting

* Remove unnecessary import

* Update src/transformers/trainer.py

Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>

* Rename AdamwBNB optimizer option

* Add training test checking that bnb memory utilization is lower

* fix merge

* fix merge; fix + extend new test

* cleanup

* expand bnb

* move all require_* candidates to testing_utils.py

Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>
Co-authored-by: Stas Bekman <stas@stason.org>
2022-04-19 16:01:29 -04:00
..
albert
Use random_attention_mask for TF tests (#16517)
2022-04-01 16:53:07 +02:00
auto
Remove parent/child tests in auto model tests (#16653)
2022-04-07 11:05:10 -04:00
bart
Reorganize file utils (#16264)
2022-03-23 10:26:33 -04:00
barthez
[Test refactor 1/5] Per-folder tests reorganization (#15725)
2022-02-23 15:46:28 -05:00
bartpho
[Test refactor 1/5] Per-folder tests reorganization (#15725)
2022-02-23 15:46:28 -05:00
beit
Clean up semantic segmentation tests (#16801)
2022-04-19 09:02:19 +02:00
benchmark
[Test refactor 1/5] Per-folder tests reorganization (#15725)
2022-02-23 15:46:28 -05:00
bert
Use random_attention_mask for TF tests (#16517)
2022-04-01 16:53:07 +02:00
bert_generation
Reorganize file utils (#16264)
2022-03-23 10:26:33 -04:00
bert_japanese
Fix issue avoid-missing-comma found at https://codereview.doctor (#16768)
2022-04-14 16:42:27 -04:00
bertweet
[Test refactor 1/5] Per-folder tests reorganization (#15725)
2022-02-23 15:46:28 -05:00
big_bird
Fix missing output_attentions in PT/Flax equivalence test (#16271)
2022-03-29 17:51:48 +02:00
bigbird_pegasus
[Test refactor 1/5] Per-folder tests reorganization (#15725)
2022-02-23 15:46:28 -05:00
blenderbot
Reorganize file utils (#16264)
2022-03-23 10:26:33 -04:00
blenderbot_small
Reorganize file utils (#16264)
2022-03-23 10:26:33 -04:00
bort
[Test refactor 1/5] Per-folder tests reorganization (#15725)
2022-02-23 15:46:28 -05:00
byt5
Fix #16660 (tokenizers setters of ids of special tokens) (#16661)
2022-04-13 07:49:06 -04:00
camembert
Reorganize file utils (#16264)
2022-03-23 10:26:33 -04:00
canine
Fix #16660 (tokenizers setters of ids of special tokens) (#16661)
2022-04-13 07:49:06 -04:00
clip
Improve test_pt_tf_model_equivalence on PT side (#16731)
2022-04-19 21:13:27 +02:00
convbert
TF: Finalize unpack_inputs-related changes (#16499)
2022-04-04 16:37:33 +01:00
convnext
Enable more test_torchscript (#16679)
2022-04-11 18:23:35 +02:00
cpm
[Test refactor 1/5] Per-folder tests reorganization (#15725)
2022-02-23 15:46:28 -05:00
ctrl
Enable more test_torchscript (#16679)
2022-04-11 18:23:35 +02:00
data2vec
[Data2Vec] Add data2vec vision (#16760)
2022-04-18 17:52:13 +02:00
deberta
Use random_attention_mask for TF tests (#16517)
2022-04-01 16:53:07 +02:00
deberta_v2
Use random_attention_mask for TF tests (#16517)
2022-04-01 16:53:07 +02:00
decision_transformer
Enable more test_torchscript (#16679)
2022-04-11 18:23:35 +02:00
deepspeed
[trainer / deepspeed] fix hyperparameter_search (#16740)
2022-04-14 17:24:38 -07:00
deit
[ViT, BEiT, DeiT, DPT] Improve code (#16799)
2022-04-18 09:25:08 -04:00
detr
Reorganize file utils (#16264)
2022-03-23 10:26:33 -04:00
distilbert
Enable more test_torchscript (#16679)
2022-04-11 18:23:35 +02:00
dit
[Tests] Fix DiT test (#16218)
2022-03-17 10:53:57 +01:00
dpr
Use random_attention_mask for TF tests (#16517)
2022-04-01 16:53:07 +02:00
dpt
[ViT, BEiT, DeiT, DPT] Improve code (#16799)
2022-04-18 09:25:08 -04:00
electra
Use random_attention_mask for TF tests (#16517)
2022-04-01 16:53:07 +02:00
encoder_decoder
Allow passing encoder_ouputs as tuple to EncoderDecoder Models (#16814)
2022-04-18 19:49:58 +02:00
extended
Add support for bitsandbytes (#15622)
2022-04-19 16:01:29 -04:00
fixtures
add a warning in SpmConverter for sentencepiece's model using the byte fallback feature (#16629)
2022-04-11 11:06:10 +02:00
flaubert
Use random_attention_mask for TF tests (#16517)
2022-04-01 16:53:07 +02:00
fnet
Enable more test_torchscript (#16679)
2022-04-11 18:23:35 +02:00
fsmt
Reorganize file utils (#16264)
2022-03-23 10:26:33 -04:00
funnel
Reduce Funnel PT/TF diff (#16744)
2022-04-13 17:19:52 +02:00
generation
Fix decoding score comparison when using logits processors or warpers (#10638)
2022-04-13 09:37:33 +01:00
glpn
Enable more test_torchscript (#16679)
2022-04-11 18:23:35 +02:00
gpt2
TF generate refactor - XLA sample (#16713)
2022-04-18 10:58:24 +01:00
gpt_neo
Reorganize file utils (#16264)
2022-03-23 10:26:33 -04:00
gptj
Use random_attention_mask for TF tests (#16517)
2022-04-01 16:53:07 +02:00
herbert
[Test refactor 1/5] Per-folder tests reorganization (#15725)
2022-02-23 15:46:28 -05:00
hubert
Enable more test_torchscript (#16679)
2022-04-11 18:23:35 +02:00
ibert
[Test refactor 1/5] Per-folder tests reorganization (#15725)
2022-02-23 15:46:28 -05:00
imagegpt
Reorganize file utils (#16264)
2022-03-23 10:26:33 -04:00
layoutlm
Use random_attention_mask for TF tests (#16517)
2022-04-01 16:53:07 +02:00
layoutlmv2
Reorganize file utils (#16264)
2022-03-23 10:26:33 -04:00
layoutxlm
Reorganize file utils (#16264)
2022-03-23 10:26:33 -04:00
led
Improve PT/TF equivalence test (#16557)
2022-04-11 22:19:12 +02:00
longformer
Some tests misusing assertTrue for comparisons fix (#16771)
2022-04-19 14:44:08 +02:00
luke
[Test refactor 1/5] Per-folder tests reorganization (#15725)
2022-02-23 15:46:28 -05:00
lxmert
Improve test_pt_tf_model_equivalence on PT side (#16731)
2022-04-19 21:13:27 +02:00
m2m_100
Reorganize file utils (#16264)
2022-03-23 10:26:33 -04:00
marian
Reorganize file utils (#16264)
2022-03-23 10:26:33 -04:00
maskformer
Enable more test_torchscript (#16679)
2022-04-11 18:23:35 +02:00
mbart
Reorganize file utils (#16264)
2022-03-23 10:26:33 -04:00
mbart50
[Test refactor 1/5] Per-folder tests reorganization (#15725)
2022-02-23 15:46:28 -05:00
megatron_bert
[Test refactor 1/5] Per-folder tests reorganization (#15725)
2022-02-23 15:46:28 -05:00
megatron_gpt2
[Test refactor 1/5] Per-folder tests reorganization (#15725)
2022-02-23 15:46:28 -05:00
mluke
[Test refactor 1/5] Per-folder tests reorganization (#15725)
2022-02-23 15:46:28 -05:00
mobilebert
Use random_attention_mask for TF tests (#16517)
2022-04-01 16:53:07 +02:00
mpnet
Enable more test_torchscript (#16679)
2022-04-11 18:23:35 +02:00
mt5
[Test refactor 1/5] Per-folder tests reorganization (#15725)
2022-02-23 15:46:28 -05:00
nystromformer
[Test refactor 1/5] Per-folder tests reorganization (#15725)
2022-02-23 15:46:28 -05:00
onnx
Add onnx export of models with a multiple choice classification head (#16758)
2022-04-19 15:51:51 +02:00
openai
Use random_attention_mask for TF tests (#16517)
2022-04-01 16:53:07 +02:00
optimization
[Test refactor 1/5] Per-folder tests reorganization (#15725)
2022-02-23 15:46:28 -05:00
pegasus
Reorganize file utils (#16264)
2022-03-23 10:26:33 -04:00
perceiver
add a test checking the format of convert_tokens_to_string's output (#16540)
2022-04-04 16:57:24 +02:00
phobert
[Test refactor 1/5] Per-folder tests reorganization (#15725)
2022-02-23 15:46:28 -05:00
pipelines
Enabling Tapex in table question answering pipeline. (#16663)
2022-04-14 09:06:14 +02:00
plbart
Reorganize file utils (#16264)
2022-03-23 10:26:33 -04:00
poolformer
Reorganize file utils (#16264)
2022-03-23 10:26:33 -04:00
prophetnet
Enable more test_torchscript (#16679)
2022-04-11 18:23:35 +02:00
qdqbert
[Test refactor 1/5] Per-folder tests reorganization (#15725)
2022-02-23 15:46:28 -05:00
rag
Reorganize file utils (#16264)
2022-03-23 10:26:33 -04:00
realm
Fix and improve REALM fine-tuning (#15297)
2022-03-03 14:10:15 +01:00
reformer
Reorganize file utils (#16264)
2022-03-23 10:26:33 -04:00
regnet
Enable more test_torchscript (#16679)
2022-04-11 18:23:35 +02:00
rembert
Use random_attention_mask for TF tests (#16517)
2022-04-01 16:53:07 +02:00
resnet
Enable more test_torchscript (#16679)
2022-04-11 18:23:35 +02:00
roberta
Use random_attention_mask for TF tests (#16517)
2022-04-01 16:53:07 +02:00
roformer
Use random_attention_mask for TF tests (#16517)
2022-04-01 16:53:07 +02:00
sagemaker
Adapt import to new structure
2022-03-24 14:40:05 -04:00
segformer
Clean up semantic segmentation tests (#16801)
2022-04-19 09:02:19 +02:00
sew
Enable more test_torchscript (#16679)
2022-04-11 18:23:35 +02:00
sew_d
[Test refactor 1/5] Per-folder tests reorganization (#15725)
2022-02-23 15:46:28 -05:00
speech_encoder_decoder
[FlaxWav2Vec2Model] Fix bug in attention mask (#16725)
2022-04-12 19:48:24 +02:00
speech_to_text
Enable more test_torchscript (#16679)
2022-04-11 18:23:35 +02:00
speech_to_text_2
[Test refactor 1/5] Per-folder tests reorganization (#15725)
2022-02-23 15:46:28 -05:00
splinter
[Test refactor 1/5] Per-folder tests reorganization (#15725)
2022-02-23 15:46:28 -05:00
squeezebert
Enable more test_torchscript (#16679)
2022-04-11 18:23:35 +02:00
swin
Enable more test_torchscript (#16679)
2022-04-11 18:23:35 +02:00
t5
TF generate refactor - XLA sample (#16713)
2022-04-18 10:58:24 +01:00
tapas
Enable more test_torchscript (#16679)
2022-04-11 18:23:35 +02:00
tapex
Add TAPEX (#16473)
2022-04-08 10:57:51 +02:00
tokenization
[Test refactor 1/5] Per-folder tests reorganization (#15725)
2022-02-23 15:46:28 -05:00
trainer
Add support for bitsandbytes (#15622)
2022-04-19 16:01:29 -04:00
transfo_xl
Enable more test_torchscript (#16679)
2022-04-11 18:23:35 +02:00
trocr
[Test refactor 1/5] Per-folder tests reorganization (#15725)
2022-02-23 15:46:28 -05:00
unispeech
Enable more test_torchscript (#16679)
2022-04-11 18:23:35 +02:00
unispeech_sat
[UniSpeechSat] correct unispeech sat (#15847)
2022-02-28 11:23:13 +01:00
utils
TF: Add sigmoid activation function (#16819)
2022-04-19 16:13:08 +01:00
van
Enable more test_torchscript (#16679)
2022-04-11 18:23:35 +02:00
vilt
Reorganize file utils (#16264)
2022-03-23 10:26:33 -04:00
vision_encoder_decoder
TF: properly handle kwargs in encoder_decoder architectures (#16465)
2022-03-29 18:17:47 +01:00
vision_text_dual_encoder
Reorganize file utils (#16264)
2022-03-23 10:26:33 -04:00
visual_bert
[Test refactor 1/5] Per-folder tests reorganization (#15725)
2022-02-23 15:46:28 -05:00
vit
[ViT, BEiT, DeiT, DPT] Improve code (#16799)
2022-04-18 09:25:08 -04:00
vit_mae
Improve test_pt_tf_model_equivalence on PT side (#16731)
2022-04-19 21:13:27 +02:00
wav2vec2
Some tests misusing assertTrue for comparisons fix (#16771)
2022-04-19 14:44:08 +02:00
wav2vec2_phoneme
Some tests misusing assertTrue for comparisons fix (#16771)
2022-04-19 14:44:08 +02:00
wav2vec2_with_lm
Some tests misusing assertTrue for comparisons fix (#16771)
2022-04-19 14:44:08 +02:00
wavlm
Enable more test_torchscript (#16679)
2022-04-11 18:23:35 +02:00
xglm
Reorganize file utils (#16264)
2022-03-23 10:26:33 -04:00
xlm
Use random_attention_mask for TF tests (#16517)
2022-04-01 16:53:07 +02:00
xlm_prophetnet
Reorganize file utils (#16264)
2022-03-23 10:26:33 -04:00
xlm_roberta
Reorganize file utils (#16264)
2022-03-23 10:26:33 -04:00
xlm_roberta_xl
[Test refactor 1/5] Per-folder tests reorganization (#15725)
2022-02-23 15:46:28 -05:00
xlnet
Use random_attention_mask for TF tests (#16517)
2022-04-01 16:53:07 +02:00
yoso
[Test refactor 1/5] Per-folder tests reorganization (#15725)
2022-02-23 15:46:28 -05:00
__init__.py
GPU text generation: mMoved the encoded_prompt to correct device
2020-01-06 15:11:12 +01:00
test_configuration_common.py
Make Transformers use cache files when hf.co is down (#16362)
2022-03-23 15:56:49 -04:00
test_feature_extraction_common.py
Make Transformers use cache files when hf.co is down (#16362)
2022-03-23 15:56:49 -04:00
test_modeling_common.py
Improve test_pt_tf_model_equivalence on PT side (#16731)
2022-04-19 21:13:27 +02:00
test_modeling_flax_common.py
[Flax] improve large model init and loading (#16148)
2022-04-19 14:19:55 +02:00
test_modeling_tf_common.py
Improve test_pt_tf_model_equivalence on PT side (#16731)
2022-04-19 21:13:27 +02:00
test_sequence_feature_extraction_common.py
Some tests misusing assertTrue for comparisons fix (#16771)
2022-04-19 14:44:08 +02:00
test_tokenization_common.py
Fix #16660 (tokenizers setters of ids of special tokens) (#16661)
2022-04-13 07:49:06 -04:00