SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity
Files
0193c8290d3487ae5de75fecba976cde9aa3590f
HuggingFace_transformer/tests
History
Lalit Pagaria 0193c8290d [RAG] Propagating of n_docs as parameter to all RagModel's related functions (#7891) 
* Propagating n_docs as parameter to all RagModel's related functions that defaults to self.config.n_docs

* Making n_docs parameter's default value to None in marginalize function

* Fixing code quality issues

* Handle the special case when generator is of T5PreTrainedModel instance type. T5PreTrainedModel do not have n_docs as parameter

* T5PreTrainedModel do not have n_docs as parameter

* Addressing review comment

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

* Correcting comment by addressing review comment

* Adding assert statement verifying that n_docs is correctly set. n_docs should be the same for both retriever and generator.

* Fixing flake8 reported issue

* Correcting test datasets for rag

* Using doc_scores instead of context_input_ids to check assert as in RagSequenceForGeneration context_input_ids can be null

* doc_scores second dimension have number of retrieved docs

* Changing assert comment

* Apply suggestions from code review

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
2020-10-19 15:15:52 +02:00
..
fixtures
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
__init__.py
GPU text generation: mMoved the encoded_prompt to correct device
2020-01-06 15:11:12 +01:00
conftest.py
ignore FutureWarning in tests (#7079)
2020-09-14 07:50:51 -04:00
test_activations_tf.py
Refactoring the TF activations functions (#7150)
2020-09-16 07:03:47 -04:00
test_activations.py
Update repo to isort v5 (#6686)
2020-08-24 11:03:01 -04:00
test_benchmark_tf.py
[Benchmarks] Change all args to from no_... to their positive form (#7075)
2020-09-23 13:25:24 -04:00
test_benchmark.py
[Benchmarks] Change all args to from no_... to their positive form (#7075)
2020-09-23 13:25:24 -04:00
test_cli.py
[transformers-cli] fix logger getter (#6777)
2020-08-27 20:01:17 -04:00
test_configuration_auto.py
Move tests/utils.py -> transformers/testing_utils.py (#5350)
2020-07-01 10:31:17 -04:00
test_configuration_common.py
Pass kwargs to configuration (#3147)
2020-03-05 17:16:57 -05:00
test_data_collator.py
Mark big downloads slow (#7325)
2020-09-22 12:21:52 -04:00
test_doc_samples.py
Move tests/utils.py -> transformers/testing_utils.py (#5350)
2020-07-01 10:31:17 -04:00
test_hf_api.py
Update repo to isort v5 (#6686)
2020-08-24 11:03:01 -04:00
test_hf_argparser.py
parse arguments from dict (#4869)
2020-07-31 04:44:23 -04:00
test_logging.py
adding TRANSFORMERS_VERBOSITY env var (#6961)
2020-09-09 04:08:01 -04:00
test_model_card.py
GPU text generation: mMoved the encoded_prompt to correct device
2020-01-06 15:11:12 +01:00
test_model_output.py
Add tests and fix various bugs in ModelOutput (#7073)
2020-09-11 12:01:33 -04:00
test_modeling_albert.py
Black 20 release
2020-08-26 17:20:22 +02:00
test_modeling_auto.py
Blenderbot (#7418)
2020-10-07 19:09:23 -04:00
test_modeling_bart.py
[tests] fix slow bart cnn test, faster marian tests (#7888)
2020-10-18 20:18:08 -04:00
test_modeling_bert_generation.py
Add "Leveraging Pretrained Checkpoints for Generation" Seq2Seq models. (#6594)
2020-09-10 16:40:51 +02:00
test_modeling_bert.py
Black 20 release
2020-08-26 17:20:22 +02:00
test_modeling_blenderbot.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_modeling_camembert.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_modeling_common.py
Blenderbot (#7418)
2020-10-07 19:09:23 -04:00
test_modeling_ctrl.py
Black 20 release
2020-08-26 17:20:22 +02:00
test_modeling_deberta.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_modeling_distilbert.py
Black 20 release
2020-08-26 17:20:22 +02:00
test_modeling_dpr.py
Black 20 release
2020-08-26 17:20:22 +02:00
test_modeling_electra.py
Black 20 release
2020-08-26 17:20:22 +02:00
test_modeling_encoder_decoder.py
clean naming (#7068)
2020-09-11 09:57:53 +02:00
test_modeling_flaubert.py
Black 20 release
2020-08-26 17:20:22 +02:00
test_modeling_fsmt.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_modeling_funnel.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_modeling_gpt2.py
Add batch inferencing support for GPT2LMHeadModel (#7552)
2020-10-14 13:40:24 +02:00
test_modeling_layoutlm.py
Add LayoutLM Model (#7064)
2020-09-22 09:28:02 -04:00
test_modeling_longformer.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_modeling_lxmert.py
fix (#6946)
2020-09-04 16:08:54 +02:00
test_modeling_marian.py
[tests] fix slow bart cnn test, faster marian tests (#7888)
2020-10-18 20:18:08 -04:00
test_modeling_mbart.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_modeling_mobilebert.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_modeling_openai.py
Gpt1 for sequence classification (#7683)
2020-10-13 05:06:15 -04:00
test_modeling_pegasus.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_modeling_rag.py
[RAG] Propagating of n_docs as parameter to all RagModel's related functions (#7891)
2020-10-19 15:15:52 +02:00
test_modeling_reformer.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_modeling_roberta.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_modeling_squeezebert.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_modeling_t5.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_modeling_tf_albert.py
Update repo to isort v5 (#6686)
2020-08-24 11:03:01 -04:00
test_modeling_tf_auto.py
Update repo to isort v5 (#6686)
2020-08-24 11:03:01 -04:00
test_modeling_tf_bert.py
Custom TF weights loading (#7422)
2020-10-05 09:58:45 -04:00
test_modeling_tf_camembert.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_modeling_tf_common.py
[Seq2Seq] Fix a couple of bugs and clean examples (#7474)
2020-10-01 17:38:50 +02:00
test_modeling_tf_ctrl.py
Black 20 release
2020-08-26 17:20:22 +02:00
test_modeling_tf_distilbert.py
test_tf_common: remove un_used mixin class parameters (#6866)
2020-09-02 10:54:40 -04:00
test_modeling_tf_electra.py
Black 20 release
2020-08-26 17:20:22 +02:00
test_modeling_tf_flaubert.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_modeling_tf_funnel.py
Fix saving TF custom models (#7291)
2020-09-22 09:31:13 -04:00
test_modeling_tf_gpt2.py
[Seq2Seq] Fix a couple of bugs and clean examples (#7474)
2020-10-01 17:38:50 +02:00
test_modeling_tf_longformer.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_modeling_tf_lxmert.py
Adding the LXMERT pretraining model (MultiModal languageXvision) to HuggingFace's suite of models (#5793)
2020-09-03 04:02:25 -04:00
test_modeling_tf_mobilebert.py
Update repo to isort v5 (#6686)
2020-08-24 11:03:01 -04:00
test_modeling_tf_openai.py
[Generate] Facilitate PyTorch generate using ModelOutputs (#6735)
2020-09-01 12:38:25 +02:00
test_modeling_tf_roberta.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_modeling_tf_t5.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_modeling_tf_transfo_xl.py
test_tf_common: remove un_used mixin class parameters (#6866)
2020-09-02 10:54:40 -04:00
test_modeling_tf_xlm_roberta.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_modeling_tf_xlm.py
Black 20 release
2020-08-26 17:20:22 +02:00
test_modeling_tf_xlnet.py
test_tf_common: remove un_used mixin class parameters (#6866)
2020-09-02 10:54:40 -04:00
test_modeling_transfo_xl.py
Black 20 release
2020-08-26 17:20:22 +02:00
test_modeling_xlm_roberta.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_modeling_xlm.py
Black 20 release
2020-08-26 17:20:22 +02:00
test_modeling_xlnet.py
[Seq2Seq] Fix a couple of bugs and clean examples (#7474)
2020-10-01 17:38:50 +02:00
test_onnx.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_optimization_tf.py
Update repo to isort v5 (#6686)
2020-08-24 11:03:01 -04:00
test_optimization.py
Format
2020-08-27 18:31:51 +02:00
test_pipelines.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_retrieval_rag.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_skip_decorators.py
remove USE_CUDA (#7861)
2020-10-19 07:08:34 -04:00
test_tokenization_albert.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_tokenization_auto.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_tokenization_bart.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_tokenization_bert_generation.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_tokenization_bert_japanese.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_tokenization_bert.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_tokenization_bertweet.py
Add new pre-trained models BERTweet and PhoBERT (#6129)
2020-09-18 13:16:43 -04:00
test_tokenization_blenderbot.py
Blenderbot (#7418)
2020-10-07 19:09:23 -04:00
test_tokenization_camembert.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_tokenization_common.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_tokenization_ctrl.py
Adding Fast tokenizers for SentencePiece based tokenizers - Breaking: remove Transfo-XL fast tokenizer (#7141)
2020-10-08 11:32:16 +02:00
test_tokenization_deberta.py
Add DeBERTa model (#5929)
2020-09-30 07:07:30 -04:00
test_tokenization_distilbert.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_tokenization_dpr.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_tokenization_fsmt.py
[ported model] FSMT (FairSeq MachineTranslation) (#6940)
2020-09-17 11:31:29 -04:00
test_tokenization_funnel.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_tokenization_gpt2.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_tokenization_herbert.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_tokenization_layoutlm.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_tokenization_lxmert.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_tokenization_marian.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_tokenization_mbart.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_tokenization_openai.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_tokenization_pegasus.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_tokenization_phobert.py
Add new pre-trained models BERTweet and PhoBERT (#6129)
2020-09-18 13:16:43 -04:00
test_tokenization_rag.py
[Rag] Fix loading of pretrained Rag Tokenizer (#7756)
2020-10-13 14:34:22 +02:00
test_tokenization_reformer.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_tokenization_roberta.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_tokenization_squeezebert.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_tokenization_t5.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_tokenization_transfo_xl.py
Adding Fast tokenizers for SentencePiece based tokenizers - Breaking: remove Transfo-XL fast tokenizer (#7141)
2020-10-08 11:32:16 +02:00
test_tokenization_utils.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_tokenization_xlm_roberta.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_tokenization_xlm.py
Adding Fast tokenizers for SentencePiece based tokenizers - Breaking: remove Transfo-XL fast tokenizer (#7141)
2020-10-08 11:32:16 +02:00
test_tokenization_xlnet.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_trainer_callback.py
fix/hide warnings (#7837)
2020-10-16 03:19:51 -04:00
test_trainer_distributed.py
Add predict step accumulation (#7767)
2020-10-14 11:41:45 -04:00
test_trainer_tpu.py
Add predict step accumulation (#7767)
2020-10-14 11:41:45 -04:00
test_trainer_utils.py
Add predict step accumulation (#7767)
2020-10-14 11:41:45 -04:00
test_trainer.py
[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)
2020-10-18 20:51:24 +02:00
test_utils_check_copies.py
Get a better error when check_copies fails (#7457)
2020-09-30 10:05:14 +02:00