SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity
6,595 Commits 5 Branches 239 Tags
3d72d47f0905772b83d02159400a7d7f9af40c4e
Commit Graph

6595 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Julien Plu
3d72d47f09 Making TF MPNet model compliant with XLA (#10260) 
* Fix XLA

* Rework cast

* Apply style
 2021-02-19 06:56:41 -05:00
Julien Plu
fb56bf2584 Making TF MobileBert model compliant with AMP (#10259) 
* Fix AMP

* Trigger CI

* Rework cast
 2021-02-19 06:55:25 -05:00
Julien Plu
2fc6284f04 Making TF Lxmert model compliant with AMP (#10257) 
* Fix AMP

* Rework cast

* Apply style
 2021-02-19 06:54:14 -05:00
Stas Bekman
d27b28d958 [ISSUES.md] propose using google colab to reproduce problems (#10270) 
* propose using google colab to reproduce problems

* Update ISSUES.md

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2021-02-18 17:15:51 -08:00
Stas Bekman
4eddc459a9 [trainer] implement support for full fp16 in evaluation/predict (#10268) 
* implement --fp16_full_eval

* Apply suggestions from code review

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* style

* add test

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2021-02-18 17:02:35 -08:00
Stas Bekman
d9a81fc0c5 fix func signature (#10271) 2021-02-18 16:44:42 -08:00
Joe Davison
c6fe17557e Script for distilling zero-shot classifier to more efficient student (#10244) 
* add zero-shot distillation script

* readme wordsmithing

* clean up code

* add multi-gpu teacher inference
plus tidying up more code

* add use_fast_tokenizer arg

* update results in readme

* more readme wordsmithing

* style

* Add handle to readme

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

* fix code block

* add error+docs about distributed & tpu

* add @sgugger format requests

* xla -> tpu

* support fp16 for teacher preds

* no checkpoint by default

* add demo colab link

* add model sharing prompt + model link

* correct resulting acc of example

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
 2021-02-18 17:08:45 -05:00
Stas Bekman
97e688bc22 [Trainer] memory tracker metrics (#10225) 
* memory tracker metrics

* go back to eval for somewhat consistency

* handle no-gpu case

* deal with stackable eval calls

* restore callback order

* style

* simplify the API

* add test

* docs

* consistently use eval_ prefix

* improve docs

* Update src/transformers/trainer_utils.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* rename method

* style

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2021-02-18 09:27:32 -08:00
Tanmay Garg
d7f38c5d1d Introduce warmup_ratio training argument (#10229) 
Introduce warmup_ratio training argument in both
TrainingArguments and TFTrainingArguments classes (#6673)
 2021-02-18 12:23:33 -05:00
Julien Plu
2acae50a0c Reduce the time spent for the TF slow tests (#10152) 
* rework savedmodel slow test

* Improve savedmodel tests

* Remove useless content
 2021-02-18 15:52:57 +01:00
Julien Plu
14ed3b978e Fix AMP (#10216) 2021-02-18 06:29:43 -05:00
Julien Plu
bdf1669e3f Making TF GPT2 compliant with XLA and AMP (#10230) 
* Fix XLA and AMP

* Fix AMP and XLA

* Apply style

* Apply Patrick's comment
 2021-02-18 09:36:01 +01:00
Stas Bekman
5da7c78ed8 update to new script; notebook notes (#10241) 2021-02-17 15:58:08 -08:00
Stas Bekman
dee876ceff [trainer] refactor place_model_on_device logic, add deepspeed (#10243) 
* refactor place_model_on_device logic, add deepspeed

* doc

* style
 2021-02-17 15:52:36 -08:00
Stas Bekman
d1eb88f42d [CI] 2 fixes (#10248) 
* fix invalid port

* missing requirements
 2021-02-17 14:12:39 -08:00
Julien Plu
7246785a67 Make TF CTRL compliant with XLA and AMP (#10209) 
* Fix XLA and AMP

* Apply style

* Remove useless cast
 2021-02-17 18:54:15 +01:00
Julien Plu
fdb2351ebb Making TF XLM-like models XLA and AMP compliant (#10211) 
* Fix Flaubert and XLM

* Remove useless cast

* Tiny fix

* Tiny fix
 2021-02-17 18:02:48 +01:00
Julien Plu
83d803ba02 Making TF BART-like models XLA and AMP compliant (#10191) 
* Update BART

* Update Blenderbot

* Update BlenderbotSmall

* Update Marian

* Update MBart

* Update MBart

* Update Pegasus

* Update template

* Fix Marian and Pegasus

* Apply style

* Default initializer

* Default initializer

* Default initializer

* Remove int32 casts

* Fix template

* Remove more cast
 2021-02-17 17:48:56 +01:00
Daniel Stancl
8d79e5ca49 Fix head masking for TFT5 (#9877) 
* Fix head_mask and decoder_head_mask in TFT5 models

* Enable test_headmasking both fot TFT5 tester
and TFT5EncoderOnly tester

Co-authored-by: patrickvonplaten <patrick.v.platen@gmail.com>
 2021-02-17 19:00:09 +03:00
Lysandre Debut
4b91965731 Factor out methods (#10215) 2021-02-17 09:53:43 -05:00
Stas Bekman
e94d63f6cb [trainer] fix ignored columns logger (#10219) 
* [trainer] fix ignored columns logger

This PR fixes a confusing log entry that says:
```
The following columns in the evaluation set don't have a corresponding argument in `T5ForConditionalGeneration.forward` and have been ignored: .
```
when everything is in order.

* Update src/transformers/trainer.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2021-02-16 13:35:39 -08:00
Joe Davison
4210cd96fc fix add_token_positions fn (#10217) 2021-02-16 14:00:05 -05:00
Sylvain Gugger
7169d1ea7b Store FLOS as floats to avoid overflow. (#10213) 2021-02-16 11:15:15 -05:00
Zhang Cheng
df1b0fb54d set tgt_lang of MBart Tokenizer for summarization (#10205) 2021-02-16 09:39:37 -05:00
Julien Plu
5c2d66a2f5 Unlock XLA test for convbert (#10207) 2021-02-16 07:59:41 -05:00
Suraj Patil
1c8c2d9ab3 [WIP][examples/seq2seq] move old s2s scripts to legacy (#10136) 
* move old s2s scripts to legacy

* add the tests back

* proper rename

* restore

* Apply suggestions from code review

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

Co-authored-by: Stas Bekman <stas@stason.org>
Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2021-02-15 10:48:02 -08:00
Stas Bekman
96897a3535 make the sub-group of tests run always (#10196) 2021-02-15 13:01:35 -05:00
Lysandre Debut
8cbd0bd137 Specify dataset dtype (#10195) 
Co-authored-by: Quentin Lhoest <lhoest.q@gmail.com>

Co-authored-by: Quentin Lhoest <lhoest.q@gmail.com>
 2021-02-15 12:57:17 -05:00
Stas Bekman
0b1f552a24 fix run_seq2seq.py; porting trainer tests to it (#10162) 
* fix run_seq2seq.py; porting DeepSpeed tests to it

* unrefactor

* defensive programming

* defensive programming 2

* port the rest of the trainer tests

* style

* a cleaner scripts dir finder

* cleanup
 2021-02-15 09:12:17 -08:00
Julien Plu
31b0560ab4 Add AMP for Albert (#10141) 2021-02-15 17:18:33 +01:00
Suraj Patil
6fc940ed09 Add mBART-50 (#10154) 
* add tokenizer for mBART-50

* update tokenizers

* make src_lang and tgt_lang optional

* update tokenizer test

* add setter

* update docs

* update conversion script

* update docs

* update conversion script

* update tokenizer

* update test

* update docs

* doc

* address Sylvain's suggestions

* fix test

* fix formatting

* nits
 2021-02-15 20:58:54 +05:30
Julien Plu
570218878a Fix TF template (#10189) 
* Fix template

* Update Seq2Seq tests
 2021-02-15 09:21:57 -05:00
Suraj Patil
2a5c990038 fix RagTokenizer (#10167) 2021-02-15 19:48:12 +05:30
Julien Plu
c8d3fa0dfd Check TF ops for ONNX compliance (#10025) 
* Add check-ops script

* Finish to implement check_tf_ops and start the test

* Make the test mandatory only for BERT

* Update tf_ops folder

* Remove useless classes

* Add the ONNX test for GPT2 and BART

* Add a onnxruntime slow test + better opset flexibility

* Fix test + apply style

* fix tests

* Switch min opset from 12 to 10

* Update src/transformers/file_utils.py

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

* Fix GPT2

* Remove extra shape_list usage

* Fix GPT2

* Address Morgan's comments

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
 2021-02-15 07:55:10 -05:00
Lysandre Debut
93bd2f7099 Add new model to labels that should not stale (#10187) 2021-02-15 06:31:29 -05:00
Nicolas Patry
900daec24e Fixing NER pipeline for list inputs. (#10184) 
Fixes #10168
 2021-02-15 06:22:45 -05:00
Sylvain Gugger
587197dcd2 Fix datasets set_format (#10178) 2021-02-15 05:49:07 -05:00
Stas Bekman
8fae93ca19 [t5 tokenizer] add info logs (#9897) 
* save fast tokenizer + add info logs

* fix tests

* remove the saving of fast tokenizer
 2021-02-13 09:10:22 -05:00
Sylvain Gugger
803498318c [Doc] Fix version control in internal pages (#10124) 2021-02-13 08:52:30 -05:00
Manuel Romero
698c9e2dbd Fix typo in comment (#10156) 2021-02-13 08:26:25 -05:00
Manuel Romero
c969366870 Fix typo in comments (#10157) 2021-02-13 08:26:01 -05:00
Nicolas Patry
c9837a0d27 Conversion from slow to fast for BPE spm vocabs contained an error. (#10120) 
* Conversion from slow to fast for BPE spm vocabs contained an error.

- There is only 1 test currently (tokenizers + slow) that used the modified path
and it's reformer, which does not contain any ids modification so the
bug was silent for now.
- The real issue is that vocab variable was overloaded by
SentencePieceExtractor, leading to Slow specific vocab oddities to be
completely ignored
- The bug was reported here https://github.com/huggingface/transformers/issues/9518
- Ran the complete tokenization test suite with slow without error
(`RUN_SLOW=1 pytest -sv tests/test_tokenization_*`)

* Remove rebase error.

* Adding the fixture.
 2021-02-13 08:24:53 -05:00
Lysandre Debut
dd3a7f9641 Revert propagation (#10171) 2021-02-13 08:19:56 -05:00
Julien Chaumond
641f418e10 [hf_api] delete deprecated methods and tests (2) 2021-02-12 21:46:17 +01:00
Julien Chaumond
eed31db948 [hf_api] delete deprecated methods and tests (#10159) 
* [hf_api] delete deprecated methods and tests

cc @lhoestq

* Update test_hf_api.py
 2021-02-12 15:35:06 -05:00
Mohamed Al Salti
1321356bdf Fix typo in GPT2DoubleHeadsModel docs (#10148) 
* Fix typo

* apply suggestion

Co-authored-by: Suraj Patil <surajp815@gmail.com>
 2021-02-12 22:48:39 +05:30
Suraj Patil
f51188cbe7 [examples/run_s2s] remove task_specific_params and update rouge computation (#10133) 
* fix rouge metrics and task specific params

* fix typo

* round metrics

* typo

* remove task_specific_params
 2021-02-12 17:18:21 +05:30
Sylvain Gugger
31245775e5 Add SageMakerTrainer for model paralellism (#10122) 
* Refactor things out of main train

* Store signature

* Add SageMakerTrainer

* Init + Copyright

* Address review comments
 2021-02-11 18:44:18 -05:00
Stas Bekman
b54cb0bd82 [DeepSpeed in notebooks] Jupyter + Colab (#10130) 
* init devices/setup explicitly

* docs + test

* simplify

* cleanup

* cleanup

* cleanup

* correct the required dist setup

* derive local_rank from env LOCAL_RANK
 2021-02-11 14:02:05 -08:00
Sylvain Gugger
6710d1d5ef Typo fix 2021-02-11 15:12:35 -05:00