SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity
Files
9a8c168f56fe3c0e21d554a577ac03beb004ef89
HuggingFace_transformer/src/transformers
History
Quentin Lhoest 9a8c168f56 Sort unique_no_split_tokens to make it deterministic (#6461) 
* change unique_no_split_tokens's type to set

* use sorted list instead of set

* style
2020-08-14 10:36:58 +02:00
..
benchmark
typo (#6225)
2020-08-04 09:31:49 -04:00
commands
Empty assert hunt (#6056)
2020-08-03 10:19:03 +02:00
data
Adding PaddingDataCollator (#6442)
2020-08-12 11:32:27 -04:00
__init__.py
MBartForConditionalGeneration (#6441)
2020-08-14 03:21:16 -04:00
activations.py
Allow gpt2 to be exported to valid ONNX (#4244)
2020-05-11 14:55:55 -04:00
configuration_albert.py
Refactor Code samples; Test code samples (#5036)
2020-06-25 16:46:00 -04:00
configuration_auto.py
MBartForConditionalGeneration (#6441)
2020-08-14 03:21:16 -04:00
configuration_bart.py
MBartForConditionalGeneration (#6441)
2020-08-14 03:21:16 -04:00
configuration_bert.py
Refactor Code samples; Test code samples (#5036)
2020-06-25 16:46:00 -04:00
configuration_camembert.py
Kill model archive maps (#4636)
2020-06-02 09:39:33 -04:00
configuration_ctrl.py
Refactor Code samples; Test code samples (#5036)
2020-06-25 16:46:00 -04:00
configuration_distilbert.py
Refactor Code samples; Test code samples (#5036)
2020-06-25 16:46:00 -04:00
configuration_dpr.py
Add DPR model (#5279)
2020-07-07 08:56:12 -04:00
configuration_electra.py
Refactor Code samples; Test code samples (#5036)
2020-06-25 16:46:00 -04:00
configuration_encoder_decoder.py
[EncoderDecoderModel] add a add_cross_attention boolean to config (#6377)
2020-08-10 19:46:48 +02:00
configuration_flaubert.py
Kill model archive maps (#4636)
2020-06-02 09:39:33 -04:00
configuration_gpt2.py
[GPT2] Correct typo in docs (#6352)
2020-08-08 20:37:29 +02:00
configuration_longformer.py
Refactor Code samples; Test code samples (#5036)
2020-06-25 16:46:00 -04:00
configuration_marian.py
Kill model archive maps (#4636)
2020-06-02 09:39:33 -04:00
configuration_mbart.py
MBartForConditionalGeneration (#6441)
2020-08-14 03:21:16 -04:00
configuration_mmbt.py
Updated Configurations
2020-01-14 14:09:09 +01:00
configuration_mobilebert.py
Refactor Code samples; Test code samples (#5036)
2020-06-25 16:46:00 -04:00
configuration_openai.py
Refactor Code samples; Test code samples (#5036)
2020-06-25 16:46:00 -04:00
configuration_pegasus.py
PegasusForConditionalGeneration (torch version) (#6340)
2020-08-11 14:31:23 -04:00
configuration_reformer.py
Feed forward chunking (#6024)
2020-08-11 03:12:45 -04:00
configuration_retribert.py
Eli5 examples (#4968)
2020-06-16 16:36:58 -04:00
configuration_roberta.py
Refactor Code samples; Test code samples (#5036)
2020-06-25 16:46:00 -04:00
configuration_t5.py
improve doc (#5185)
2020-06-22 19:00:11 +02:00
configuration_transfo_xl.py
Refactor Code samples; Test code samples (#5036)
2020-06-25 16:46:00 -04:00
configuration_utils.py
Feed forward chunking (#6024)
2020-08-11 03:12:45 -04:00
configuration_xlm_roberta.py
Kill model archive maps (#4636)
2020-06-02 09:39:33 -04:00
configuration_xlm.py
Refactor Code samples; Test code samples (#5036)
2020-06-25 16:46:00 -04:00
configuration_xlnet.py
fix (#6257)
2020-08-05 07:37:57 -04:00
convert_albert_original_tf_checkpoint_to_pytorch.py
Add AlbertForPreTraining and TFAlbertForPreTraining models. (#4057)
2020-05-07 19:44:51 -04:00
convert_bart_original_pytorch_checkpoint_to_pytorch.py
mBART Conversion script (#6230)
2020-08-04 09:53:51 -04:00
convert_bert_original_tf2_checkpoint_to_pytorch.py
Add script to convert tf2.x checkpoint to PyTorch (#5791)
2020-08-03 03:53:38 -04:00
convert_bert_original_tf_checkpoint_to_pytorch.py
Remove __future__ imports.
2019-12-22 17:47:54 +01:00
convert_bert_pytorch_checkpoint_to_original_tf.py
GPU text generation: mMoved the encoded_prompt to correct device
2020-01-06 15:11:12 +01:00
convert_dialogpt_original_pytorch_checkpoint_to_pytorch.py
[dialogpt] conversion script
2020-03-10 15:09:29 -04:00
convert_dpr_original_checkpoint_to_pytorch.py
Add DPR model (#5279)
2020-07-07 08:56:12 -04:00
convert_electra_original_tf_checkpoint_to_pytorch.py
ELECTRA (#3257)
2020-04-03 14:10:54 -04:00
convert_gpt2_original_tf_checkpoint_to_pytorch.py
Use built-in open().
2019-12-22 18:38:56 +01:00
convert_graph_to_onnx.py
Enable ONNX/ONNXRuntime optimizations through converter script (#6131)
2020-07-31 09:45:13 +02:00
convert_longformer_original_pytorch_lightning_to_pytorch.py
[Longformer For Question Answering] Conversion script, doc, small fixes (#4593)
2020-05-26 14:58:47 +02:00
convert_marian_to_pytorch.py
Empty assert hunt (#6056)
2020-08-03 10:19:03 +02:00
convert_mbart_original_checkpoint_to_pytorch.py
mBART Conversion script (#6230)
2020-08-04 09:53:51 -04:00
convert_mobilebert_original_tf_checkpoint_to_pytorch.py
Add MobileBert (#4901)
2020-06-19 16:38:36 -04:00
convert_openai_original_tf_checkpoint_to_pytorch.py
Use built-in open().
2019-12-22 18:38:56 +01:00
convert_pegasus_tf_to_pytorch.py
PegasusForConditionalGeneration (torch version) (#6340)
2020-08-11 14:31:23 -04:00
convert_pytorch_checkpoint_to_tf2.py
Imports organization
2020-07-06 10:27:10 -04:00
convert_reformer_trax_checkpoint_to_pytorch.py
[Reformer] Add Enwiki8 Reformer Model - Adapt convert script (#4282)
2020-05-11 16:38:07 +02:00
convert_roberta_original_pytorch_checkpoint_to_pytorch.py
Update to match renamed attributes in fairseq master (#5972)
2020-08-05 07:23:55 -04:00
convert_t5_original_tf_checkpoint_to_pytorch.py
Remove __future__ imports.
2019-12-22 17:47:54 +01:00
convert_transfo_xl_original_tf_checkpoint_to_pytorch.py
Use built-in open().
2019-12-22 18:38:56 +01:00
convert_xlm_original_pytorch_checkpoint_to_pytorch.py
Use built-in open().
2019-12-22 18:38:56 +01:00
convert_xlnet_original_tf_checkpoint_to_pytorch.py
Remove __future__ imports.
2019-12-22 17:47:54 +01:00
file_utils.py
Tf model outputs (#6247)
2020-08-05 11:34:39 -04:00
generation_tf_utils.py
Fix docs and bad word tokens generation_utils.py (#6387)
2020-08-13 13:12:16 +02:00
generation_utils.py
[EncoderDecoder] Add Cross Attention for GPT2 (#6415)
2020-08-14 09:43:29 +02:00
hf_api.py
[HfApi] Add support for pipeline_tag
2020-06-24 16:54:00 +00:00
hf_argparser.py
Fix #6428 (#6437)
2020-08-12 08:47:30 -04:00
integrations.py
Adds comet_ml to the list of auto-experiment loggers (#6176)
2020-08-06 11:31:30 -04:00
modelcard.py
Pipelines: miscellanea of QoL improvements and small features... (#4632)
2020-06-03 03:51:31 -04:00
modeling_albert.py
Fix FFN dropout in TFAlbertLayer, and split dropout in TFAlbertAttent… (#4323)
2020-08-12 07:52:42 -04:00
modeling_auto.py
MBartForConditionalGeneration (#6441)
2020-08-14 03:21:16 -04:00
modeling_bart.py
fix consistency CrossEntropyLoss in modeling_bart (#6265)
2020-08-07 17:44:28 +08:00
modeling_bert.py
[EncoderDecoder] Add encoder-decoder for roberta/ vanilla longformer (#6411)
2020-08-12 18:23:30 +02:00
modeling_camembert.py
Switch from return_tuple to return_dict (#6138)
2020-07-30 09:17:00 -04:00
modeling_ctrl.py
Switch from return_tuple to return_dict (#6138)
2020-07-30 09:17:00 -04:00
modeling_distilbert.py
Switch from return_tuple to return_dict (#6138)
2020-07-30 09:17:00 -04:00
modeling_dpr.py
Switch from return_tuple to return_dict (#6138)
2020-07-30 09:17:00 -04:00
modeling_electra.py
Fix the tests for Electra (#6284)
2020-08-07 09:30:57 -04:00
modeling_encoder_decoder.py
[EncoderDecoder] Add Cross Attention for GPT2 (#6415)
2020-08-14 09:43:29 +02:00
modeling_flaubert.py
Fix FlauBERT GPU test (#6142)
2020-07-30 11:11:48 -04:00
modeling_gpt2.py
[EncoderDecoder] Add Cross Attention for GPT2 (#6415)
2020-08-14 09:43:29 +02:00
modeling_longformer.py
TF Longformer (#5764)
2020-08-10 23:25:06 +02:00
modeling_marian.py
rename prepare_translation_batch -> prepare_seq2seq_batch (#6103)
2020-08-11 15:57:07 -04:00
modeling_mbart.py
MBartForConditionalGeneration (#6441)
2020-08-14 03:21:16 -04:00
modeling_mmbt.py
Switch from return_tuple to return_dict (#6138)
2020-07-30 09:17:00 -04:00
modeling_mobilebert.py
Tf model outputs (#6247)
2020-08-05 11:34:39 -04:00
modeling_openai.py
Empty assert hunt (#6056)
2020-08-03 10:19:03 +02:00
modeling_outputs.py
Remove outdated BERT tips (#6217)
2020-08-04 01:17:56 +08:00
modeling_pegasus.py
PegasusForConditionalGeneration (torch version) (#6340)
2020-08-11 14:31:23 -04:00
modeling_reformer.py
fix for pytorch < 1.6 (#6300)
2020-08-06 21:14:46 +02:00
modeling_retribert.py
[tokenizers] Updates data processors, docstring, examples and model cards to the new API (#5308)
2020-06-26 19:48:14 +02:00
modeling_roberta.py
[EncoderDecoder] Add encoder-decoder for roberta/ vanilla longformer (#6411)
2020-08-12 18:23:30 +02:00
modeling_t5.py
Tf model outputs (#6247)
2020-08-05 11:34:39 -04:00
modeling_tf_albert.py
Fix FFN dropout in TFAlbertLayer, and split dropout in TFAlbertAttent… (#4323)
2020-08-12 07:52:42 -04:00
modeling_tf_auto.py
TF Longformer (#5764)
2020-08-10 23:25:06 +02:00
modeling_tf_bert.py
[EncoderDecoder] Add encoder-decoder for roberta/ vanilla longformer (#6411)
2020-08-12 18:23:30 +02:00
modeling_tf_camembert.py
Tf model outputs (#6247)
2020-08-05 11:34:39 -04:00
modeling_tf_ctrl.py
Tf model outputs (#6247)
2020-08-05 11:34:39 -04:00
modeling_tf_distilbert.py
Tf model outputs (#6247)
2020-08-05 11:34:39 -04:00
modeling_tf_electra.py
Tf model outputs (#6247)
2020-08-05 11:34:39 -04:00
modeling_tf_flaubert.py
Tf model outputs (#6247)
2020-08-05 11:34:39 -04:00
modeling_tf_gpt2.py
added n_inner argument to gpt2 config (#6296)
2020-08-06 17:47:32 +02:00
modeling_tf_longformer.py
Test model outputs equivalence (#6445)
2020-08-13 11:59:35 -04:00
modeling_tf_mobilebert.py
Tf model outputs (#6247)
2020-08-05 11:34:39 -04:00
modeling_tf_openai.py
Tf model outputs (#6247)
2020-08-05 11:34:39 -04:00
modeling_tf_outputs.py
Tf model outputs (#6247)
2020-08-05 11:34:39 -04:00
modeling_tf_pytorch_utils.py
add another e.g. to avoid confusion (#6055)
2020-07-30 08:53:35 -04:00
modeling_tf_roberta.py
Tf model outputs (#6247)
2020-08-05 11:34:39 -04:00
modeling_tf_t5.py
Tf model outputs (#6247)
2020-08-05 11:34:39 -04:00
modeling_tf_transfo_xl_utilities.py
Fix saved model creation (#5468)
2020-08-03 08:10:40 -04:00
modeling_tf_transfo_xl.py
Tf model outputs (#6247)
2020-08-05 11:34:39 -04:00
modeling_tf_utils.py
Update modeling_tf_utils.py (#6372)
2020-08-10 02:55:11 -04:00
modeling_tf_xlm_roberta.py
Tf model outputs (#6247)
2020-08-05 11:34:39 -04:00
modeling_tf_xlm.py
Tf model outputs (#6247)
2020-08-05 11:34:39 -04:00
modeling_tf_xlnet.py
Tf model outputs (#6247)
2020-08-05 11:34:39 -04:00
modeling_transfo_xl_utilities.py
Shift labels internally within TransfoXLLMHeadModel when called with labels (#3716)
2020-04-13 18:11:23 +02:00
modeling_transfo_xl.py
Empty assert hunt (#6056)
2020-08-03 10:19:03 +02:00
modeling_utils.py
typo fix (#6462)
2020-08-13 09:36:48 -04:00
modeling_xlm_roberta.py
Switch from return_tuple to return_dict (#6138)
2020-07-30 09:17:00 -04:00
modeling_xlm.py
Patch GPU failures (#6281)
2020-08-07 02:58:15 -04:00
modeling_xlnet.py
Tf model outputs (#6247)
2020-08-05 11:34:39 -04:00
optimization_tf.py
Allow to set Adam beta1, beta2 in TrainingArgs (#5592)
2020-07-27 05:31:37 -04:00
optimization.py
lr_schedulers: add get_polynomial_decay_schedule_with_warmup (#6361)
2020-08-11 17:56:41 -04:00
pipelines.py
add targets arg to fill-mask pipeline (#6239)
2020-08-12 12:48:29 -04:00
testing_utils.py
testing utils: capturing std streams context manager (#6231)
2020-08-11 03:56:47 -04:00
tokenization_albert.py
[tokenizers] Updates data processors, docstring, examples and model cards to the new API (#5308)
2020-06-26 19:48:14 +02:00
tokenization_auto.py
MBartForConditionalGeneration (#6441)
2020-08-14 03:21:16 -04:00
tokenization_bart.py
MBartForConditionalGeneration (#6441)
2020-08-14 03:21:16 -04:00
tokenization_bert_japanese.py
Replace mecab-python3 with fugashi for Japanese tokenization (#6086)
2020-07-31 04:41:14 -04:00
tokenization_bert.py
Add strip_accents to basic BertTokenizer. (#6280)
2020-08-06 18:52:28 +08:00
tokenization_camembert.py
[tokenizers] Updates data processors, docstring, examples and model cards to the new API (#5308)
2020-06-26 19:48:14 +02:00
tokenization_ctrl.py
Cleanup fast tokenizers integration (#3706)
2020-04-18 13:43:57 +02:00
tokenization_distilbert.py
Cleanup fast tokenizers integration (#3706)
2020-04-18 13:43:57 +02:00
tokenization_dpr.py
Fix tests imports dpr (#5576)
2020-07-07 16:35:12 +02:00
tokenization_electra.py
Cleanup fast tokenizers integration (#3706)
2020-04-18 13:43:57 +02:00
tokenization_flaubert.py
Fix CI after killing archive maps (#4724)
2020-06-02 10:21:09 -04:00
tokenization_gpt2.py
Fix fast tokenizers too (#5562)
2020-07-06 18:45:01 -04:00
tokenization_longformer.py
[Longformer] more models + model cards (#4628)
2020-05-28 11:11:05 +02:00
tokenization_marian.py
rename prepare_translation_batch -> prepare_seq2seq_batch (#6103)
2020-08-11 15:57:07 -04:00
tokenization_mbart.py
MBartForConditionalGeneration (#6441)
2020-08-14 03:21:16 -04:00
tokenization_mobilebert.py
Add MobileBert (#4901)
2020-06-19 16:38:36 -04:00
tokenization_openai.py
GPT2 tokenizer should not output token type IDs (#5546)
2020-07-06 11:33:57 -04:00
tokenization_pegasus.py
rename prepare_translation_batch -> prepare_seq2seq_batch (#6103)
2020-08-11 15:57:07 -04:00
tokenization_reformer.py
Refactor Code samples; Test code samples (#5036)
2020-06-25 16:46:00 -04:00
tokenization_retribert.py
Eli5 examples (#4968)
2020-06-16 16:36:58 -04:00
tokenization_roberta.py
Fix fast tokenizers too (#5562)
2020-07-06 18:45:01 -04:00
tokenization_t5.py
Update tokenization_t5.py (#5717)
2020-07-14 00:02:03 -04:00
tokenization_transfo_xl.py
Empty assert hunt (#6056)
2020-08-03 10:19:03 +02:00
tokenization_utils_base.py
Fixes to make life easier with the nlp library (#6423)
2020-08-12 08:00:56 -04:00
tokenization_utils_fast.py
Doc tokenizer (#6110)
2020-07-30 14:51:19 -04:00
tokenization_utils.py
Sort unique_no_split_tokens to make it deterministic (#6461)
2020-08-14 10:36:58 +02:00
tokenization_xlm_roberta.py
[tokenizers] Updates data processors, docstring, examples and model cards to the new API (#5308)
2020-06-26 19:48:14 +02:00
tokenization_xlm.py
[tokenizers] Updates data processors, docstring, examples and model cards to the new API (#5308)
2020-06-26 19:48:14 +02:00
tokenization_xlnet.py
[tokenizers] Updates data processors, docstring, examples and model cards to the new API (#5308)
2020-06-26 19:48:14 +02:00
trainer_tf.py
Move prediction_loss_only to TrainingArguments (#6426)
2020-08-12 08:03:45 -04:00
trainer_utils.py
Adds comet_ml to the list of auto-experiment loggers (#6176)
2020-08-06 11:31:30 -04:00
trainer.py
Move prediction_loss_only to TrainingArguments (#6426)
2020-08-12 08:03:45 -04:00
training_args_tf.py
Trainer + wandb quality of life logging tweaks (#6241)
2020-08-05 09:05:52 -04:00
training_args.py
Move prediction_loss_only to TrainingArguments (#6426)
2020-08-12 08:03:45 -04:00