SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity
6,945 Commits 5 Branches 239 Tags
6c40e49712972141c3d7aeba4ed90bb79f2bb078
Commit Graph

1530 Commits

Author SHA1 Message Date
Andrea Cappelli
6c40e49712 Run mlm pad to multiple for fp16 (#11128) 
* Add mlm collator pad to multiple option (#10627)

* Use padding to 8x in run mlm (#10627)
 2021-04-08 16:12:49 -04:00
Stas Bekman
c6d664849b [DeepSpeed] ZeRO Stage 3 (#10753) 
* synced gpus

* fix

* fix

* need to use t5-small for quality tests

* notes

* complete merge

* fix a disappearing std stream problem

* start zero3 tests

* wip

* tune params

* sorting out the pre-trained model loading

* reworking generate loop wip

* wip

* style

* fix tests

* split the tests

* refactor tests

* wip

* parameterized

* fix

* workout the resume from non-ds checkpoint pass + test

* cleanup

* remove no longer needed code

* split getter/setter functions

* complete the docs

* suggestions

* gpus and their compute capabilities link

* Apply suggestions from code review

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

* style

* remove invalid paramgd

* automatically configure zero3 params that rely on hidden size

* make _get_resized_embeddings zero3-aware

* add test exercising resize_token_embeddings()

* add docstring

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
 2021-04-08 09:53:01 -07:00
Stas Bekman
acc851e1ff [run_clm] clarify why we get the tokenizer warning on long input (#11145) 
* clarify why we get the warning here

* Update examples/language-modeling/run_clm.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* wording

* style

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2021-04-08 09:46:28 -07:00
Stas Bekman
424419f549 [examples] fix white space (#11099) 
these get concatenated without whitespace, so fix it
 2021-04-07 09:20:58 -04:00
Stas Bekman
c9035e4537 fix: The 'warn' method is deprecated (#11105) 
* The 'warn' method is deprecated

* fix test
 2021-04-07 09:20:06 -04:00
Sylvain Gugger
fd338abdeb Style 2021-04-06 19:54:13 -04:00
SHYAM SUNDER KUMAR
aef4cf8c52 accelerate question answering examples with no trainer (#11091) 
* accelerate question answering examples with no trainer

* removed train and eval flags also fixed fill np array function

* Update examples/question-answering/run_qa_beam_search_no_trainer.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update examples/question-answering/run_qa_no_trainer.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2021-04-06 19:35:21 -04:00
Lysandre
9853c5dd58 Development on v4.6.0dev0 2021-04-06 12:53:25 -04:00
Lysandre
4906a29f7f Release v4.5.0
Some checks failed
Release - Conda / build_and_package (push) Has been cancelled
Details
2021-04-06 12:37:47 -04:00
Hemil Desai
6ab7d1a429 Add Readme for language modeling scripts with accelerate (#11073) 2021-04-05 20:56:12 -04:00
Hemil Desai
b51b87c41d Add examples/language_modeling/run_clm_no_trainer.py (#11026) 
* Initial draft for clm no trainer

* Remove unwanted args

* Fix bug

* Update examples/language-modeling/run_clm_no_trainer.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2021-04-05 12:27:52 -04:00
Stas Bekman
3d39226a51 s|Pretrained|PreTrained| (#11048) 2021-04-04 18:08:42 -07:00
versis
335c0ca35c fixed typo: logging instead of logger (#11025) 2021-04-02 09:22:22 -04:00
Hemil Desai
838f83d84c Add examples/language_modeling/run_mlm_no_trainer.py (#11001) 
* Add initial script for finetuning MLM models with accelerate

* Add evaluation metric calculation

* Fix bugs

* Use no_grad on evaluation

* update script docstring

* Update examples/language-modeling/run_mlm_no_trainer.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* PR feedback

* Fix CI failure

* Update examples/language-modeling/run_mlm_no_trainer.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2021-03-31 18:49:45 -04:00
Sylvain Gugger
acc3bd9d2a Enforce string-formatting with f-strings (#10980) 
* First third

* Styling and fix mistake

* Quality

* All the rest

* Treat %s and %d

* typo

* Missing )

* Apply suggestions from code review

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
 2021-03-31 10:00:27 -04:00
WybeKoper
645f45c462 Fixed some typos and removed legacy url (#10989) 
* Fixed typos

* Removed legacy colab notebook from readme

Co-authored-by: WybeKoper <WybeKoper@users.noreply.github.com>
 2021-03-31 16:53:15 +05:30
Yih-Dar
e031162a6b fix md file to avoid evaluation crash (#10962) 2021-03-30 21:26:22 +03:00
Philipp Schmid
3e09d813aa [examples/s2s] added py7zr dep (#10971) 
* added py7zr

* comment out check_min for sagemaker test

* added min version again
 2021-03-30 23:17:12 +05:30
Stas Bekman
05c966f24b [vulnerability] dep fix (#10954) 
Fixes https://github.com/huggingface/transformers/security/dependabot/examples/research_projects/lxmert/requirements.txt/Pygments/open

@LysandreJik
 2021-03-29 17:25:47 -04:00
Daniel Stancl
5057213bcc Add examples/multiple-choice/run_swag_no_trainer.py (#10934) 
* Initial commit

* Another bunch of updates

* make style quliaty + delete debug arg from bash script

* Use compue_metrics func

* Do a few fixes

* Add copyright

* Fix typos
 2021-03-29 16:41:09 -04:00
Sylvain Gugger
4002f95eb6 Remove duplicate code 2021-03-29 15:27:12 -04:00
Daniel Stancl
d7b50ce469 Add examples/run_ner_no_trainer.py (#10902) 
* Add NER example with accelerate library

* This commit contains the first (yet really unfinished)
version of a script for showing how to train HuggingFace model
with their new accelerate library.

* Fix metric calculation

* make style quality

* mv ner_no_trainer to token-classification dir

* Delete --debug flag from running script

* hf_datasets -> raw_datasets

* Make a few slight adjustments

* Add an informative comment + rewrite a help comment

* Change header

* Fix a few things

* Enforce to use fast tokenizers only

* DataCollatorWithPadding -> DataCollatorForTokenClassification

* Change bash script: python3 -> accelerate launch

* make style

* Add a few missing things (see below)

* Add a max-lenghth padding to predictions and labels to
enable accelerate gather functionality

* Add PyTorch no trainer example to the example README.md

* Remove --do-train from args as being redundant for now

* DataCollatorWithPadding -> DataCollatorForTokenClassification

* Remove some obsolete args.do_train conditions from the script

* Delete --do_train from bash running script

* Delete use_slow_tokenizer from args

* Add unintentionally removed flag --label_all_tokens

* Delete --debug flag from running script
 2021-03-29 15:11:23 -04:00
WybeKoper
ddea8771c6 Updated colab links in readme of examples (#10932) 
Co-authored-by: WybeKoper <WybeKoper@users.noreply.github.com>
 2021-03-29 08:47:09 -04:00
Bhadresh Savani
4f21e1ddd6 fixed finename (#10939) 2021-03-28 09:48:12 -07:00
Stas Bekman
3c27d246e5 [vulnerability] fix dependency (#10914) 
this PR fixes https://github.com/huggingface/transformers/security/dependabot/examples/research_projects/lxmert/requirements.txt/PyYAML/open
 2021-03-26 09:06:11 -04:00
Jethro Kuan
5f1491d3b3 run_glue_no_trainer: datasets -> raw_datasets (#10898) 
Use the correct variable (raw_datasets) instead of the module (datasets)
where appropriate.
 2021-03-25 08:28:17 -04:00
Bhadresh Savani
7ef40120a0 [Examples] Added predict stage and Updated Example Template (#10868) 
* added predict stage

* added test keyword in exception message

* removed example specific saving predictions

* fixed f-string error

* removed extra line

Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>

Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>
 2021-03-23 10:37:59 -07:00
Eliza Szczechla
9f8fa4e973 Use DataCollatorForSeq2Seq in run_summarization in all cases (#10856) 
Co-authored-by: Eliza <eliza@habanero.tiger.com.pl>
 2021-03-22 15:05:39 -04:00
Boris Dayma
125ccead71 feat(wandb): logging and configuration improvements (#10826) 
* feat: ensure unique artifact id

* feat: allow manual init

* fix: simplify reinit logic

* fix: no dropped value + immediate commits

* fix: wandb use in sagemaker

* docs: improve documenation and formatting

* fix: typos

* docs: improve formatting
 2021-03-22 10:45:17 -04:00
Stas Bekman
8fb4671811 [vulnerability] in example deps fix (#10817) 
Takes care of:
https://github.com/huggingface/transformers/security/dependabot/examples/research_projects/lxmert/requirements.txt/jinja2/open

@LysandreJik

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
 2021-03-22 09:05:24 -04:00
dependabot[bot]
dbfe379514 Bump jinja2 from 2.11.2 to 2.11.3 in /examples/research_projects/lxmert (#10818) 
Bumps [jinja2](https://github.com/pallets/jinja) from 2.11.2 to 2.11.3.
- [Release notes](https://github.com/pallets/jinja/releases)
- [Changelog](https://github.com/pallets/jinja/blob/master/CHANGES.rst)
- [Commits](https://github.com/pallets/jinja/compare/2.11.2...2.11.3)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-03-22 08:54:50 -04:00
Qiushi Pan
29904a967b Update FINE_TUNE_XLSR_WAV2VEC2.md (#10849) 
Fix typo.
 2021-03-22 07:58:59 -04:00
Patrick von Platen
0f226f78ce push (#10846) 2021-03-22 10:32:21 +03:00
Suraj Patil
82b8d8c7b0 Update FINE_TUNE_XLSR_WAV2VEC2.md 2021-03-21 22:47:09 +05:30
Patrick von Platen
af6125ffdb Update FINE_TUNE_XLSR_WAV2VEC2.md 2021-03-21 12:31:33 +03:00
Patrick von Platen
5aaf6e1460 small improvements for wav2vec2 info script (#10829) 2021-03-21 11:41:44 +03:00
Suraj Patil
68b55885ed add doc for Local machine (#10828) 2021-03-21 13:25:34 +05:30
Julien Chaumond
1438c487df wav2vec doc tweaks (#10808) 
* wording/typos tweaks

* Make model upload instructions simpler
 2021-03-19 12:48:54 -04:00
Patrick von Platen
b9570a813c Update FINE_TUNE_XLSR_WAV2VEC2.md 2021-03-19 19:45:28 +03:00
Sylvain Gugger
946400fb68 Expand a bit the presentation of examples (#10799) 
* Expand a bit the presentation of examples

* Apply suggestions from code review

Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>

* Address review comments

Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>
 2021-03-19 10:06:08 -04:00
Bhadresh Savani
fd1d9f1ab8 [Example] Updating Question Answering examples for Predict Stage (#10792) 
* added prediction stage and eval fix

* style correction

* removed extra lines
 2021-03-19 09:42:17 -04:00
Patrick von Platen
e8968bd03a [XLSR-Wav2Vec2 Info doc] Add a couple of lines (#10806) 
* finish

* fix

* fix

* fix

* fix
 2021-03-19 12:52:54 +03:00
Stas Bekman
427ea3fecb addressing vulnerability report in research project deps (#10802) 
Following up on a security alert:
https://github.com/huggingface/transformers/security/dependabot/examples/research_projects/lxmert/requirements.txt/Pillow/open
 2021-03-18 22:02:10 -04:00
Patrick von Platen
2ae678229f Update FINE_TUNE_XLSR_WAV2VEC2.md 2021-03-19 00:29:20 +03:00
Patrick von Platen
68a3215949 Update FINE_TUNE_XLSR_WAV2VEC2.md 2021-03-19 00:27:40 +03:00
Patrick von Platen
03df3fbcb4 Update FINE_TUNE_XLSR_WAV2VEC2.md 2021-03-19 00:26:49 +03:00
Patrick von Platen
e84adbed40 Add XLSR-Wav2Vec2 Fine-Tuning README.md (#10786) 
* upload

* upload fine-tuning script

* improve

* adapt

* Apply suggestions from code review

* correct

* upload

* finalize

* remove @

* correct typos
 2021-03-19 00:22:43 +03:00
Stas Bekman
9352b5151a [examples/seq2seq/README.md] fix t5 examples (#10734) 
* [examples/seq2seq] fix t5 examples

This PR:
* fixes T5 examples to include `--source_prefix` - it's **not** optional. If you give it a try you will see that you get 10x worse bleu scores w/o it. w/ `27.6849`, w/ `2.374`
* added a normal translation example w/o the peculiarities of MBart and T5
* reduces the default max samples to 50 so it's much faster to test quickly

summarization seems to be broken for t5 score-wise: https://github.com/huggingface/transformers/issues/10733

@sgugger

* specify explicitly the t5 models requiring the special handling

* one more

* update the t5 summarization example to use cnn_dailymail

* move max*samples into the top level README.md

* better wording

* better wording
 2021-03-18 09:55:39 -07:00
Julien Chaumond
4f3e93cfaf [file_utils] do not gobble certain kinds of requests.ConnectionError (#10235) 
* do not gobble certain kinds of requests.ConnectionError

* Apply review comments

Co-authored-by: Lysandre <lysandre.debut@reseau.eseo.fr>
 2021-03-18 12:37:45 -04:00
Suraj Patil
5f19c07a70 add run_common_voice script (#10767) 
* add initial script

* finish script

* add shell script example

* accept chars_to_ignor as cl arg

* align the script with other example scripts

* add torchaudio dep
 2021-03-18 17:21:16 +05:30