SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity
Files
a1bbcf3f6c20e15fe799a8659d6b7bd36fdf11ed
HuggingFace_transformer/tests
History
Patrick von Platen a1bbcf3f6c Refactoring the generate() function (#6949) 
* first draft

* show design proposition for new generate method

* up

* make better readable

* make first version

* gpt2 tests pass

* make beam search for gpt2 work

* add first encoder-decoder code

* delete typo

* make t5 work

* save indermediate

* make bart work with beam search

* finish beam search bart / t5

* add default kwargs

* make more tests pass

* fix no bad words sampler

* some fixes and tests for all distribution processors

* fix test

* fix rag slow tests

* merge to master

* add nograd to generate

* make all slow tests pass

* speed up generate

* fix edge case bug

* small fix

* correct typo

* add type hints and docstrings

* fix typos in tests

* add beam search tests

* add tests for beam scorer

* fix test rag

* finish beam search tests

* move generation tests in seperate file

* fix generation tests

* more tests

* add aggressive generation tests

* fix tests

* add gpt2 sample test

* add more docstring

* add more docs

* finish doc strings

* apply some more of sylvains and sams comments

* fix some typos

* make fix copies

* apply lysandres and sylvains comments

* final corrections on examples

* small fix for reformer
2020-11-03 16:04:22 +01:00
..
fixtures
New run glue script (#7917)
2020-10-22 11:42:22 -04:00
__init__.py
GPU text generation: mMoved the encoded_prompt to correct device
2020-01-06 15:11:12 +01:00
conftest.py
[CI] generate separate report files as artifacts (#7995)
2020-10-27 09:25:07 -04:00
test_activations_tf.py
Replace swish with silu (#8166)
2020-10-30 15:09:10 -04:00
test_activations.py
Replace swish with silu (#8166)
2020-10-30 15:09:10 -04:00
test_benchmark_tf.py
[Benchmarks] Change all args to from no_... to their positive form (#7075)
2020-09-23 13:25:24 -04:00
test_benchmark.py
[Benchmarks] Change all args to from no_... to their positive form (#7075)
2020-09-23 13:25:24 -04:00
test_cli.py
[transformers-cli] fix logger getter (#6777)
2020-08-27 20:01:17 -04:00
test_configuration_auto.py
Move tests/utils.py -> transformers/testing_utils.py (#5350)
2020-07-01 10:31:17 -04:00
test_configuration_common.py
[PretrainedConfig] Fix save pretrained config for edge case (#7943)
2020-10-22 15:39:01 +02:00
test_data_collator.py
Fix label name in DataCollatorForNextSentencePrediction test (#8048)
2020-10-26 09:23:12 -04:00
test_doc_samples.py
Fix ignore list behavior in doctests (#8213)
2020-11-02 08:47:37 -05:00
test_flax_auto.py
Integrate Bert-like model on Flax runtime. (#3722)
2020-10-19 09:55:41 -04:00
test_generation_beam_search.py
Refactoring the generate() function (#6949)
2020-11-03 16:04:22 +01:00
test_generation_logits_process.py
Refactoring the generate() function (#6949)
2020-11-03 16:04:22 +01:00
test_generation_utils.py
Refactoring the generate() function (#6949)
2020-11-03 16:04:22 +01:00
test_hf_api.py
Update repo to isort v5 (#6686)
2020-08-24 11:03:01 -04:00
test_hf_argparser.py
Smarter prediction loop and no- -> no_ in console args (#8151)
2020-10-29 10:56:25 -04:00
test_logging.py
Fix doc errors and typos across the board (#8139)
2020-10-29 10:33:33 -04:00
test_model_card.py
GPU text generation: mMoved the encoded_prompt to correct device
2020-01-06 15:11:12 +01:00
test_model_output.py
Add tests and fix various bugs in ModelOutput (#7073)
2020-09-11 12:01:33 -04:00
test_modeling_albert.py
Black 20 release
2020-08-26 17:20:22 +02:00
test_modeling_auto.py
Blenderbot (#7418)
2020-10-07 19:09:23 -04:00
test_modeling_bart.py
Refactoring the generate() function (#6949)
2020-11-03 16:04:22 +01:00
test_modeling_bert_generation.py
Refactoring the generate() function (#6949)
2020-11-03 16:04:22 +01:00
test_modeling_bert.py
Refactoring the generate() function (#6949)
2020-11-03 16:04:22 +01:00
test_modeling_blenderbot.py
Refactoring the generate() function (#6949)
2020-11-03 16:04:22 +01:00
test_modeling_camembert.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_modeling_common.py
Refactoring the generate() function (#6949)
2020-11-03 16:04:22 +01:00
test_modeling_ctrl.py
Refactoring the generate() function (#6949)
2020-11-03 16:04:22 +01:00
test_modeling_deberta.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_modeling_distilbert.py
Black 20 release
2020-08-26 17:20:22 +02:00
test_modeling_dpr.py
Black 20 release
2020-08-26 17:20:22 +02:00
test_modeling_electra.py
Black 20 release
2020-08-26 17:20:22 +02:00
test_modeling_encoder_decoder.py
ProphetNet (#7157)
2020-10-19 17:36:09 +02:00
test_modeling_flaubert.py
Black 20 release
2020-08-26 17:20:22 +02:00
test_modeling_flax_bert.py
Integrate Bert-like model on Flax runtime. (#3722)
2020-10-19 09:55:41 -04:00
test_modeling_flax_roberta.py
Integrate Bert-like model on Flax runtime. (#3722)
2020-10-19 09:55:41 -04:00
test_modeling_fsmt.py
Refactoring the generate() function (#6949)
2020-11-03 16:04:22 +01:00
test_modeling_funnel.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_modeling_gpt2.py
Refactoring the generate() function (#6949)
2020-11-03 16:04:22 +01:00
test_modeling_layoutlm.py
[testing] rename skip targets + docs (#7863)
2020-10-20 04:39:13 -04:00
test_modeling_longformer.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_modeling_lxmert.py
fix (#6946)
2020-09-04 16:08:54 +02:00
test_modeling_marian.py
TFMarian, TFMbart, TFPegasus, TFBlenderbot (#7987)
2020-10-30 11:23:16 -04:00
test_modeling_mbart.py
Cleanup pytorch tests (#8033)
2020-10-26 08:59:06 -04:00
test_modeling_mobilebert.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_modeling_openai.py
Refactoring the generate() function (#6949)
2020-11-03 16:04:22 +01:00
test_modeling_pegasus.py
Cleanup pytorch tests (#8033)
2020-10-26 08:59:06 -04:00
test_modeling_prophetnet.py
Refactoring the generate() function (#6949)
2020-11-03 16:04:22 +01:00
test_modeling_rag.py
Ci test tf super slow (#8007)
2020-10-30 10:25:48 -04:00
test_modeling_reformer.py
Refactoring the generate() function (#6949)
2020-11-03 16:04:22 +01:00
test_modeling_roberta.py
Refactoring the generate() function (#6949)
2020-11-03 16:04:22 +01:00
test_modeling_squeezebert.py
Ci test tf super slow (#8007)
2020-10-30 10:25:48 -04:00
test_modeling_t5.py
Refactoring the generate() function (#6949)
2020-11-03 16:04:22 +01:00
test_modeling_tf_albert.py
Update repo to isort v5 (#6686)
2020-08-24 11:03:01 -04:00
test_modeling_tf_auto.py
Update repo to isort v5 (#6686)
2020-08-24 11:03:01 -04:00
test_modeling_tf_bart.py
TFMarian, TFMbart, TFPegasus, TFBlenderbot (#7987)
2020-10-30 11:23:16 -04:00
test_modeling_tf_bert.py
Custom TF weights loading (#7422)
2020-10-05 09:58:45 -04:00
test_modeling_tf_blenderbot.py
TFMarian, TFMbart, TFPegasus, TFBlenderbot (#7987)
2020-10-30 11:23:16 -04:00
test_modeling_tf_camembert.py
Ci test tf super slow (#8007)
2020-10-30 10:25:48 -04:00
test_modeling_tf_common.py
Ci test tf super slow (#8007)
2020-10-30 10:25:48 -04:00
test_modeling_tf_ctrl.py
Black 20 release
2020-08-26 17:20:22 +02:00
test_modeling_tf_distilbert.py
test_tf_common: remove un_used mixin class parameters (#6866)
2020-09-02 10:54:40 -04:00
test_modeling_tf_electra.py
Black 20 release
2020-08-26 17:20:22 +02:00
test_modeling_tf_flaubert.py
Ci test tf super slow (#8007)
2020-10-30 10:25:48 -04:00
test_modeling_tf_funnel.py
Fix saving TF custom models (#7291)
2020-09-22 09:31:13 -04:00
test_modeling_tf_gpt2.py
[Seq2Seq] Fix a couple of bugs and clean examples (#7474)
2020-10-01 17:38:50 +02:00
test_modeling_tf_longformer.py
Ci test tf super slow (#8007)
2020-10-30 10:25:48 -04:00
test_modeling_tf_lxmert.py
Ci test tf super slow (#8007)
2020-10-30 10:25:48 -04:00
test_modeling_tf_marian.py
TFMarian, TFMbart, TFPegasus, TFBlenderbot (#7987)
2020-10-30 11:23:16 -04:00
test_modeling_tf_mbart.py
TFMarian, TFMbart, TFPegasus, TFBlenderbot (#7987)
2020-10-30 11:23:16 -04:00
test_modeling_tf_mobilebert.py
Ci test tf super slow (#8007)
2020-10-30 10:25:48 -04:00
test_modeling_tf_openai.py
[Generate] Facilitate PyTorch generate using ModelOutputs (#6735)
2020-09-01 12:38:25 +02:00
test_modeling_tf_pegasus.py
TFMarian, TFMbart, TFPegasus, TFBlenderbot (#7987)
2020-10-30 11:23:16 -04:00
test_modeling_tf_pytorch.py
[tests|tokenizers] Refactoring pipelines test backbone - Small tokenizers improvements - General tests speedups (#7970)
2020-10-23 15:58:19 +02:00
test_modeling_tf_roberta.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_modeling_tf_t5.py
Ci test tf super slow (#8007)
2020-10-30 10:25:48 -04:00
test_modeling_tf_transfo_xl.py
test_tf_common: remove un_used mixin class parameters (#6866)
2020-09-02 10:54:40 -04:00
test_modeling_tf_xlm_roberta.py
Ci test tf super slow (#8007)
2020-10-30 10:25:48 -04:00
test_modeling_tf_xlm.py
Black 20 release
2020-08-26 17:20:22 +02:00
test_modeling_tf_xlnet.py
test_tf_common: remove un_used mixin class parameters (#6866)
2020-09-02 10:54:40 -04:00
test_modeling_transfo_xl.py
Refactoring the generate() function (#6949)
2020-11-03 16:04:22 +01:00
test_modeling_xlm_prophetnet.py
Ci test tf super slow (#8007)
2020-10-30 10:25:48 -04:00
test_modeling_xlm_roberta.py
Ci test tf super slow (#8007)
2020-10-30 10:25:48 -04:00
test_modeling_xlm.py
Refactoring the generate() function (#6949)
2020-11-03 16:04:22 +01:00
test_modeling_xlnet.py
Refactoring the generate() function (#6949)
2020-11-03 16:04:22 +01:00
test_onnx.py
Ci test tf super slow (#8007)
2020-10-30 10:25:48 -04:00
test_optimization_tf.py
Update repo to isort v5 (#6686)
2020-08-24 11:03:01 -04:00
test_optimization.py
Format
2020-08-27 18:31:51 +02:00
test_pipelines_common.py
Fix the behaviour of DefaultArgumentHandler (removing it). (#8180)
2020-11-02 12:33:50 +01:00
test_pipelines_conversational.py
[tests|tokenizers] Refactoring pipelines test backbone - Small tokenizers improvements - General tests speedups (#7970)
2020-10-23 15:58:19 +02:00
test_pipelines_dialog.py
[tests|tokenizers] Refactoring pipelines test backbone - Small tokenizers improvements - General tests speedups (#7970)
2020-10-23 15:58:19 +02:00
test_pipelines_feature_extraction.py
[tests|tokenizers] Refactoring pipelines test backbone - Small tokenizers improvements - General tests speedups (#7970)
2020-10-23 15:58:19 +02:00
test_pipelines_fill_mask.py
Fix the behaviour of DefaultArgumentHandler (removing it). (#8180)
2020-11-02 12:33:50 +01:00
test_pipelines_ner.py
[tests|tokenizers] Refactoring pipelines test backbone - Small tokenizers improvements - General tests speedups (#7970)
2020-10-23 15:58:19 +02:00
test_pipelines_question_answering.py
[tests|tokenizers] Refactoring pipelines test backbone - Small tokenizers improvements - General tests speedups (#7970)
2020-10-23 15:58:19 +02:00
test_pipelines_sentiment_analysis.py
[tests|tokenizers] Refactoring pipelines test backbone - Small tokenizers improvements - General tests speedups (#7970)
2020-10-23 15:58:19 +02:00
test_pipelines_summarization.py
[tests|tokenizers] Refactoring pipelines test backbone - Small tokenizers improvements - General tests speedups (#7970)
2020-10-23 15:58:19 +02:00
test_pipelines_text2text_generation.py
[tests|tokenizers] Refactoring pipelines test backbone - Small tokenizers improvements - General tests speedups (#7970)
2020-10-23 15:58:19 +02:00
test_pipelines_text_generation.py
[tests|tokenizers] Refactoring pipelines test backbone - Small tokenizers improvements - General tests speedups (#7970)
2020-10-23 15:58:19 +02:00
test_pipelines_translation.py
[tests|tokenizers] Refactoring pipelines test backbone - Small tokenizers improvements - General tests speedups (#7970)
2020-10-23 15:58:19 +02:00
test_pipelines_zero_shot.py
Fix the behaviour of DefaultArgumentHandler (removing it). (#8180)
2020-11-02 12:33:50 +01:00
test_retrieval_rag.py
Allow Custom Dataset in RAG Retriever (#7763)
2020-10-19 19:42:45 +02:00
test_skip_decorators.py
[testing] rename skip targets + docs (#7863)
2020-10-20 04:39:13 -04:00
test_tokenization_albert.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_tokenization_auto.py
[testing] slow tests should be marked as slow (#7895)
2020-10-22 06:34:05 -04:00
test_tokenization_bart.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_tokenization_bert_generation.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_tokenization_bert_japanese.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_tokenization_bert.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_tokenization_bertweet.py
Add new pre-trained models BERTweet and PhoBERT (#6129)
2020-09-18 13:16:43 -04:00
test_tokenization_blenderbot.py
Fix + Test (#8049)
2020-10-26 12:32:27 -04:00
test_tokenization_camembert.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_tokenization_common.py
[tokenizers] Fixing #8001 - Adding tests on tokenizers serialization (#8006)
2020-10-26 10:27:48 +01:00
test_tokenization_ctrl.py
Adding Fast tokenizers for SentencePiece based tokenizers - Breaking: remove Transfo-XL fast tokenizer (#7141)
2020-10-08 11:32:16 +02:00
test_tokenization_deberta.py
Add DeBERTa model (#5929)
2020-09-30 07:07:30 -04:00
test_tokenization_distilbert.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_tokenization_dpr.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_tokenization_fsmt.py
fsmt slow test uses lists (#8031)
2020-10-26 08:32:36 -04:00
test_tokenization_funnel.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_tokenization_gpt2.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_tokenization_herbert.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_tokenization_layoutlm.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_tokenization_lxmert.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_tokenization_marian.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_tokenization_mbart.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_tokenization_openai.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_tokenization_pegasus.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_tokenization_phobert.py
Add new pre-trained models BERTweet and PhoBERT (#6129)
2020-09-18 13:16:43 -04:00
test_tokenization_prophetnet.py
ProphetNet (#7157)
2020-10-19 17:36:09 +02:00
test_tokenization_rag.py
[Rag] Fix loading of pretrained Rag Tokenizer (#7756)
2020-10-13 14:34:22 +02:00
test_tokenization_reformer.py
[Reformer] remove reformer pad_token_id (#7991)
2020-10-23 10:29:15 -04:00
test_tokenization_roberta.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_tokenization_squeezebert.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_tokenization_t5.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_tokenization_transfo_xl.py
Adding Fast tokenizers for SentencePiece based tokenizers - Breaking: remove Transfo-XL fast tokenizer (#7141)
2020-10-08 11:32:16 +02:00
test_tokenization_utils.py
Fix BatchEncoding.word_to_tokens for removed tokens (#7939)
2020-10-23 10:29:37 -04:00
test_tokenization_xlm_prophetnet.py
ProphetNet (#7157)
2020-10-19 17:36:09 +02:00
test_tokenization_xlm_roberta.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_tokenization_xlm.py
Adding Fast tokenizers for SentencePiece based tokenizers - Breaking: remove Transfo-XL fast tokenizer (#7141)
2020-10-08 11:32:16 +02:00
test_tokenization_xlnet.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_trainer_callback.py
Fix a bug for CallbackHandler.callback_list (#8052)
2020-10-27 10:37:04 -04:00
test_trainer_distributed.py
[testing] port test_trainer_distributed to distributed pytest + TestCasePlus enhancements (#8107)
2020-10-28 11:51:32 -04:00
test_trainer_tpu.py
Add predict step accumulation (#7767)
2020-10-14 11:41:45 -04:00
test_trainer_utils.py
Add predict step accumulation (#7767)
2020-10-14 11:41:45 -04:00
test_trainer.py
TensorBoard/Wandb/optuna/raytune integration improvements. (#7935)
2020-10-21 17:18:52 +02:00
test_utils_check_copies.py
Get a better error when check_copies fails (#7457)
2020-09-30 10:05:14 +02:00