SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity
6,698 Commits 5 Branches 239 Tags
f284089ec4cf6da2d659512b09f2cf526dc6b308
Commit Graph

244 Commits

Author SHA1 Message Date
Mehrad Moradshahi
b880508440 tokenization_marian.py: use current_spm for decoding (#10357) 
* Fix Marian decoding

Tokenizer's decode and batch_decode now accepts a new argument (use_source_tokenizer) which indicates whether the source spm should be used to decode ids. This is useful for Marian models specificallly when decoding source input ids.

* Adapt docstrings

Co-authored-by: Sylvain Gugger <sylvain.gugger@gmail.com>
 2021-03-08 08:14:31 -05:00
Suraj Patil
2a737bffef [M2M100] fix positional embeddings (#10590) 
* fix tests

* emb should be a parameter

* fix positional embeddings

* fix make_weights

* don't save pos embeds

* add comment to describe the clamping
 2021-03-08 16:06:19 +05:30
Oren Amsalem
d59464db6b fix BART Summarization example in doc (#10582) 2021-03-08 15:45:06 +05:30
Yu
9dd054fba2 fix tf doc bug (#10570) 2021-03-07 22:31:50 -05:00
Suraj Patil
f6e74a63ca Add m2m100 (#10236) 
* m2m_100

* no layernorm_embedding

* sinusoidal positional embeddings

* update pos embeddings

* add default config values

* tokenizer

* add conversion script

* fix config

* fix pos embed

* remove _float_tensor

* update tokenizer

* update lang codes

* handle lang codes

* fix pos embeds

* fix spm key

* put embedding weights on device

* remove qa and seq classification heads

* fix convert script

* lang codes pn one line

* fix embeds

* fix tokenizer

* fix tokenizer

* add fast tokenizer

* style

* M2M100MT => M2M100

* fix copyright, style

* tokenizer converter

* vocab file

* remove fast tokenizer

* fix embeds

* fix tokenizer

* fix tests

* add tokenizer tests

* add integration test

* quality

* fix model name

* fix test

* doc

* doc

* fix doc

* add copied from statements

* fix tokenizer tests

* apply review suggestions

* fix urls

* fix shift_tokens_right

* apply review suggestions

* fix

* fix doc

* add lang code to id

* remove unused function

* update checkpoint names

* fix copy

* fix tokenizer

* fix checkpoint names

* fix merge issue

* style
 2021-03-06 22:14:16 +05:30
Daniel Hug
90ecc29656 Refactoring checkpoint names for multiple models (#10527) 
* Refactor checkpoint name in ALBERT and ALBERT_tf

* Refactor checkpoint name in BART and BART_tf

* Refactor checkpoint name in BERT generation

* Refactor checkpoint name in Blenderbot_tf

* Refactor checkpoint name in Blenderbot_small_tf

* Refactor checkpoint name in ConvBERT AND CONVBERT_TF

* Refactor checkpoint name in CTRL AND CTRL_TF

* Refactor checkpoint name in DistilBERT AND DistilBERT_TF

* Refactor checkpoint name in DistilBERT redo

* Refactor checkpoint name in Electra and Electra_tf

* Refactor checkpoint name in FlauBERT and FlauBERT_tf

* Refactor checkpoint name in FSMT

* Refactor checkpoint name in GPT2 and GPT2_tf

* Refactor checkpoint name in IBERT

* Refactor checkpoint name in LED and LED_tf

* Refactor checkpoint name in Longformer and Longformer_tf

* Refactor checkpoint name in Lxmert and Lxmert_tf

* Refactor checkpoint name in Marian_tf

* Refactor checkpoint name in MBART and MBART_tf

* Refactor checkpoint name in MobileBERT and MobileBERT_tf

* Refactor checkpoint name in mpnet and mpnet_tf

* Refactor checkpoint name in openai and openai_tf

* Refactor checkpoint name in pegasus_tf

* Refactor checkpoint name in reformer

* Refactor checkpoint name in Roberta and Roberta_tf

* Refactor checkpoint name in SqueezeBert

* Refactor checkpoint name in Transformer_xl and Transformer_xl_tf

* Refactor checkpoint name in XLM and XLM_tf

* Refactor checkpoint name in XLNET and XLNET_tf

* Refactor checkpoint name in BERT_tf

* run make tests, style, quality, fixup
 2021-03-05 18:06:55 -05:00
Sylvain Gugger
7da995c00c Fix embeddings for PyTorch 1.8 (#10549) 
* Fix embeddings for PyTorch 1.8

* Try with PyTorch 1.8.0

* Fix embeddings init

* Fix copies

* Typo

* More typos
 2021-03-05 16:18:48 -05:00
Chen Liang
3e056c1003 Typo correction. (#10531) 
DEBERTA_PRETRAINED_MODEL_ARCHIVE_LIST => DEBERTA_V2_PRETRAINED_MODEL_ARCHIVE_LIST in line 31.
 2021-03-05 15:27:09 -05:00
Patrick von Platen
c503a1c15e [ProphetNet] Bart-like Refactor (#10501) 
* first step to refactor

* make all fast tests pass

* make all slow tests pass

* save intermediate

* correct cache

* finish PR

* make fp16 work
 2021-03-04 23:27:12 +03:00
felixgwu
d064fb5647 Fix the bug in constructing the all_hidden_states of DeBERTa v2 (#10466) 
* fix all_hidden_states

* use output_states instead of next_kv
 2021-03-03 12:05:21 -05:00
Sylvain Gugger
801ff969ce Refactor checkpoint name in BERT and MobileBERT (#10424) 
* Refactor checkpoint name in BERT and MobileBERT

* Add option to check copies

* Add QuestionAnswering

* Add last models

* Make black happy
 2021-03-03 11:21:17 -05:00
Patrick von Platen
2d2ed2cc18 [T5] Fix speed degradation bug t5 (#10496) 
* fix speed degradation bug t5

* fix for all models

* fix code quality
 2021-03-03 12:42:41 +03:00
WybeKoper
5dc303e281 Fixed minor spelling mistakes (#10489) 
Co-authored-by: WybeKoper <WybeKoper@users.noreply.github.com>
 2021-03-03 14:17:25 +05:30
Suraj Patil
a106bde5a7 [Wav2Vec2FeatureExtractor] smal fixes (#10455) 
* smal fixes

* don't check for None
 2021-03-01 20:19:52 +05:30
Patrick von Platen
11655fafdd remove feature extraction config (#10457) 2021-03-01 12:30:12 +03:00
Patrick von Platen
0234de8418 Add Fine-Tuning for Wav2Vec2 (#10145) 
* add encode labels function to tokenizer

* start adding finetuning

* init dropout

* upload

* correct convert script

* apply changes

* fix second typo

* make first dummy training run

* adapt convert script

* push confg for comparison

* remove conf

* finish training

* adapt data collator

* add research folder

* update according to fairseq feedback

* some minor corrections

* refactor masking indices a bit

* some minor changes

* clean tokenizer

* finish clean-up

* remove previous logic

* update run script

* correct training

* finish changes

* finish model

* correct bug

* fix training a bit more

* add some tests

* finish gradient checkpointing

* finish example

* correct gradient checkpointing

* improve tokenization method

* revert changes in tokenizer

* revert general change

* adapt fine-tuning

* update

* save intermediate test

* Update README.md

* finish finetuning

* delete conversion script

* Update src/transformers/models/wav2vec2/configuration_wav2vec2.py

* Update src/transformers/models/wav2vec2/processing_wav2vec2.py

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

* finish wav2vec2 script

* finish wav2vec2 fine-tuning

* finalize test

* correct test

* adapt tests

* finish

* remove test file

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
 2021-03-01 12:13:17 +03:00
Patrick von Platen
d03695f3a2 [LED] Correct Docs (#10419) 
* correct docs

* correct tf model docs as well
 2021-02-26 17:53:28 +03:00
Lysandre Debut
88cc26dcd1 Ignore unexpected weights from PT conversion (#10397) 2021-02-25 10:42:27 -05:00
Sehoon Kim
63645b3b11 I-BERT model support (#10153) 
* IBertConfig, IBertTokentizer added

* IBert Model names moified

* tokenizer bugfix

* embedding -> QuantEmbedding

* quant utils added

* quant_mode added to configuration

* QuantAct added, Embedding layer + QuantAct addition

* QuantAct added

* unused path removed, QKV quantized

* self attention layer all quantized, except softmax

* temporarl commit

* all liner layers quantized

* quant_utils bugfix

* bugfix: requantization missing

* IntGELU added

* IntSoftmax added

* LayerNorm implemented

* LayerNorm implemented all

* names changed: roberta->ibert

* config not inherit from ROberta

* No support for CausalLM

* static quantization added, quantize_model.py removed

* import modules uncommented

* copyrights fixed

* minor bugfix

* quant_modules, quant_utils merged as one file

* import * fixed

* unused runfile removed

* make style run

* configutration.py docstring fixed

* refactoring: comments removed, function name fixed

* unused dependency removed

* typo fixed

* comments(Copied from), assertion string added

* refactoring: super(..) -> super(), etc.

* refactoring

* refarctoring

* make style

* refactoring

* cuda -> to(x.device)

* weight initialization removed

* QuantLinear set_param removed

* QuantEmbedding set_param removed

* IntLayerNorm set_param removed

* assert string added

* assertion error message fixed

* is_decoder removed

* enc-dec arguments/functions removed

* Converter removed

* quant_modules docstring fixed

* conver_slow_tokenizer rolled back

* quant_utils docstring fixed

* unused aruments e.g. use_cache removed from config

* weight initialization condition fixed

* x_min, x_max initialized with small values to avoid div-zero exceptions

* testing code for ibert

* test emb, linear, gelu, softmax added

* test ln and act added

* style reformatted

* force_dequant added

* error tests overrided

* make style

* Style + Docs

* force dequant tests added

* Fix fast tokenizer in init

* Fix doc

* Remove space

* docstring, IBertConfig, chunk_size

* test_modeling_ibert refactoring

* quant_modules.py refactoring

* e2e integration test added

* tokenizers removed

* IBertConfig added to tokenizer_auto.py

* bugfix

* fix docs & test

* fix style num 2

* final fixes

Co-authored-by: Sehoon Kim <sehoonkim@berkeley.edu>
Co-authored-by: Lysandre <lysandre.debut@reseau.eseo.fr>
Co-authored-by: Sylvain Gugger <sylvain.gugger@gmail.com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
 2021-02-25 10:06:42 -05:00
Patrick von Platen
cb38ffcc5e [PretrainedFeatureExtractor] + Wav2Vec2FeatureExtractor, Wav2Vec2Processor, Wav2Vec2Tokenizer (#10324) 
* push to show

* small improvement

* small improvement

* Update src/transformers/feature_extraction_utils.py

* Update src/transformers/feature_extraction_utils.py

* implement base

* add common tests

* make all tests pass for wav2vec2

* make padding work & add more tests

* finalize feature extractor utils

* add call method to feature extraction

* finalize feature processor

* finish tokenizer

* finish general processor design

* finish tests

* typo

* remove bogus file

* finish docstring

* add docs

* finish docs

* small fix

* correct docs

* save intermediate

* load changes

* apply changes

* apply changes to doc

* change tests

* apply surajs recommend

* final changes

* Apply suggestions from code review

* fix typo

* fix import

* correct docstring
 2021-02-25 17:42:46 +03:00
mingruimingrui
894db6701e Bugfix: Removal of padding_idx in BartLearnedPositionalEmbedding (#10200) 
* Assumption of padding_idx <2 might not stand

* Use offset instead of 2

* Fix with black

* Change behavior to warning instead for backward compatibility.

* Fix with black

* Remove warning

* Make padding_idx non-required

* padding_idx fix for blenderbot

* padding_idx fix for blenderbot_small

* padding_idx fix for led

* padding_idx fix for mbart

* Remove extra whitespaces

* padding_idx fix for template

* Fix padding_idx passed to nn.Embedding mistake

* Fixed padding_idx passed to positional embedding in template

* Remove padding_idx from pytorch learned positional embeddings

* Remove accidentally added quotes

* Remove padding_idx from tf learned positional embeddings

* Remove zeroing of weights in __init__

Co-authored-by: Wang Ming Rui <mingrui.wang@C02CJTUYMD6M.local>
 2021-02-25 14:33:13 +03:00
Julien Plu
cdcdd5f03a Rework casts (#10274) 2021-02-24 08:38:29 -05:00
abhishek thakur
2d458b2c7d ConvBERT fix torch <> tf weights conversion (#10314) 
* convbert conversion test

* fin

* fin

* fin

* clean up tf<->pt conversion

* remove from_pt

Co-authored-by: patrickvonplaten <patrick.v.platen@gmail.com>
 2021-02-24 14:55:34 +03:00
Julien Chaumond
4a1ab7cb6c [bert-base-german-cased] cp to hardcoded urls (#10353) 2021-02-23 12:30:47 -05:00
Sylvain Gugger
9e147d31f6 Deprecate prepare_seq2seq_batch (#10287) 
* Deprecate prepare_seq2seq_batch

* Fix last tests

* Apply suggestions from code review

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
Co-authored-by: Suraj Patil <surajp815@gmail.com>

* More review comments

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
Co-authored-by: Suraj Patil <surajp815@gmail.com>
 2021-02-22 12:36:16 -05:00
Julien Plu
19e737b93e Making TF Longformer-like models compliant with AMP (#10233) 
* AMP

* Add LED

* Apply style

* Fix longformer
 2021-02-22 15:41:56 +01:00
tagucci
88605f37a6 fix typo in conversion script (#10316) 
* fix typo in conversion script

* style

Co-authored-by: Stas Bekman <stas@stason.org>
 2021-02-21 07:54:27 -08:00
Pengcheng He
9a7e63729f Integrate DeBERTa v2(the 1.5B model surpassed human performance on Su… (#10018) 
* Integrate DeBERTa v2(the 1.5B model surpassed human performance on SuperGLUE); Add DeBERTa v2 900M,1.5B models;

* DeBERTa-v2

* Fix v2 model loading issue (#10129)

* Doc members

* Update src/transformers/models/deberta/modeling_deberta.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Address Sylvain's comments

* Address Patrick's comments

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

* Style

Co-authored-by: Lysandre <lysandre.debut@reseau.eseo.fr>
Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
 2021-02-19 18:34:44 -05:00
Julien Plu
34df26ec3a Making TF OpenAI GPT model compliant with AMP and XLA (#10261) 
* Fix AMP and XLA

* Remove useless var
 2021-02-19 09:33:25 -05:00
Julien Plu
3e116ed331 Making TF TransfoXL model compliant with AMP (#10264) 
* Fix AMP

* Apply style

* Remove unused import
 2021-02-19 06:58:07 -05:00
Julien Plu
86caeb7636 Fix XLA and AMP (#10262) 2021-02-19 06:57:16 -05:00
Julien Plu
3d72d47f09 Making TF MPNet model compliant with XLA (#10260) 
* Fix XLA

* Rework cast

* Apply style
 2021-02-19 06:56:41 -05:00
Julien Plu
fb56bf2584 Making TF MobileBert model compliant with AMP (#10259) 
* Fix AMP

* Trigger CI

* Rework cast
 2021-02-19 06:55:25 -05:00
Julien Plu
2fc6284f04 Making TF Lxmert model compliant with AMP (#10257) 
* Fix AMP

* Rework cast

* Apply style
 2021-02-19 06:54:14 -05:00
Julien Plu
14ed3b978e Fix AMP (#10216) 2021-02-18 06:29:43 -05:00
Julien Plu
bdf1669e3f Making TF GPT2 compliant with XLA and AMP (#10230) 
* Fix XLA and AMP

* Fix AMP and XLA

* Apply style

* Apply Patrick's comment
 2021-02-18 09:36:01 +01:00
Julien Plu
7246785a67 Make TF CTRL compliant with XLA and AMP (#10209) 
* Fix XLA and AMP

* Apply style

* Remove useless cast
 2021-02-17 18:54:15 +01:00
Julien Plu
fdb2351ebb Making TF XLM-like models XLA and AMP compliant (#10211) 
* Fix Flaubert and XLM

* Remove useless cast

* Tiny fix

* Tiny fix
 2021-02-17 18:02:48 +01:00
Julien Plu
83d803ba02 Making TF BART-like models XLA and AMP compliant (#10191) 
* Update BART

* Update Blenderbot

* Update BlenderbotSmall

* Update Marian

* Update MBart

* Update MBart

* Update Pegasus

* Update template

* Fix Marian and Pegasus

* Apply style

* Default initializer

* Default initializer

* Default initializer

* Remove int32 casts

* Fix template

* Remove more cast
 2021-02-17 17:48:56 +01:00
Daniel Stancl
8d79e5ca49 Fix head masking for TFT5 (#9877) 
* Fix head_mask and decoder_head_mask in TFT5 models

* Enable test_headmasking both fot TFT5 tester
and TFT5EncoderOnly tester

Co-authored-by: patrickvonplaten <patrick.v.platen@gmail.com>
 2021-02-17 19:00:09 +03:00
Julien Plu
31b0560ab4 Add AMP for Albert (#10141) 2021-02-15 17:18:33 +01:00
Suraj Patil
6fc940ed09 Add mBART-50 (#10154) 
* add tokenizer for mBART-50

* update tokenizers

* make src_lang and tgt_lang optional

* update tokenizer test

* add setter

* update docs

* update conversion script

* update docs

* update conversion script

* update tokenizer

* update test

* update docs

* doc

* address Sylvain's suggestions

* fix test

* fix formatting

* nits
 2021-02-15 20:58:54 +05:30
Suraj Patil
2a5c990038 fix RagTokenizer (#10167) 2021-02-15 19:48:12 +05:30
Julien Plu
c8d3fa0dfd Check TF ops for ONNX compliance (#10025) 
* Add check-ops script

* Finish to implement check_tf_ops and start the test

* Make the test mandatory only for BERT

* Update tf_ops folder

* Remove useless classes

* Add the ONNX test for GPT2 and BART

* Add a onnxruntime slow test + better opset flexibility

* Fix test + apply style

* fix tests

* Switch min opset from 12 to 10

* Update src/transformers/file_utils.py

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

* Fix GPT2

* Remove extra shape_list usage

* Fix GPT2

* Address Morgan's comments

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
 2021-02-15 07:55:10 -05:00
Stas Bekman
8fae93ca19 [t5 tokenizer] add info logs (#9897) 
* save fast tokenizer + add info logs

* fix tests

* remove the saving of fast tokenizer
 2021-02-13 09:10:22 -05:00
Mohamed Al Salti
1321356bdf Fix typo in GPT2DoubleHeadsModel docs (#10148) 
* Fix typo

* apply suggestion

Co-authored-by: Suraj Patil <surajp815@gmail.com>
 2021-02-12 22:48:39 +05:30
Patrick von Platen
495c157d6f [Wav2Vec2] Improve Tokenizer & Model for batched inference (#10117) 
* save intermediate

* finish batch the same as fairseq

* add normalization

* fix batched input

* add better comment

* Update src/transformers/models/wav2vec2/modeling_wav2vec2.py

* add nice docstring

* add tokenizer tests

* make all slow tests pass

* finish PR

* correct import
 2021-02-11 15:40:54 +03:00
Suraj Patil
c130e67dce remove adjust_logits_during_generation method (#10087) 
* add forced logits processors

* delete adjust_logits method

* add forced_eos_token_id argument in config

* add tests for forced logits processors

* update gen utils tests

* add forced option to tf generate

* remove adjust_logits method from tf models

* update adjust_logits for marian

* delete _force_token_id_to_be_generated method

* style

* import warnings

* pass max_length to _get_logits_processor

* set forced_eos_token_id to None

* set forced attributes in conf utils

* typo

* fix rag generate

* add forced_eos_token_id in rag config

* remove force_bos_token_to_be_generated from BartConfig

* remove _force_token_ids_generation from FSMT

* nit

* fix negative constant

* apply suggestions from code review
 2021-02-10 22:39:09 +05:30
Julien Plu
22a32cf485 Fix TF LED/Longformer attentions computation (#10007) 
* Fix test

* Remove commented test

* Fix name

* Apply style

* Fix check copies

* Remove prints

* Restore boolean

* Fix reshape
 2021-02-10 10:58:37 -05:00
Suraj Patil
3e0c62b611 [RAG] fix generate (#10094) 
* fix rag generate and tests

* put back adjust_logits_during_generation

* tests are okay

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
 2021-02-09 21:57:38 +03:00