SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity
6,631 Commits 5 Branches 239 Tags
cb38ffcc5e0ae2fac653342ac36dc75c15ea178f
Commit Graph

1449 Commits

Author SHA1 Message Date
Patrick von Platen
cb38ffcc5e [PretrainedFeatureExtractor] + Wav2Vec2FeatureExtractor, Wav2Vec2Processor, Wav2Vec2Tokenizer (#10324) 
* push to show

* small improvement

* small improvement

* Update src/transformers/feature_extraction_utils.py

* Update src/transformers/feature_extraction_utils.py

* implement base

* add common tests

* make all tests pass for wav2vec2

* make padding work & add more tests

* finalize feature extractor utils

* add call method to feature extraction

* finalize feature processor

* finish tokenizer

* finish general processor design

* finish tests

* typo

* remove bogus file

* finish docstring

* add docs

* finish docs

* small fix

* correct docs

* save intermediate

* load changes

* apply changes

* apply changes to doc

* change tests

* apply surajs recommend

* final changes

* Apply suggestions from code review

* fix typo

* fix import

* correct docstring
 2021-02-25 17:42:46 +03:00
Stas Bekman
3437d12134 [Trainer/Deepspeed] handle get_last_lr() before first step() (#10362) 
* handle get_last_lr() before first step()

* abstract away the lr getting logic

* cleanup

* add test

* move to utils
 2021-02-23 17:42:25 -08:00
Akmal
23e87c27be Fix broken examples/seq2seq/README.md markdown (#10344) 2021-02-23 10:49:25 -05:00
Stas Bekman
622a8c5995 [trainer] add Trainer methods for metrics logging and saving (#10266) 
* make logging and saving trainer built-in

* Update src/transformers/trainer.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2021-02-22 13:02:53 -08:00
Stas Bekman
eab0afc19c [Trainer] implement gradient_accumulation_steps support in DeepSpeed integration (#10310) 
* implement gradient_accumulation_steps support in DeepSpeed integration

* typo

* cleanup

* cleanup
 2021-02-22 11:15:59 -08:00
Stas Bekman
f991daed18 defensive programming + expand/correct README (#10295) 2021-02-22 10:58:50 -08:00
Julien Plu
536aee99bb Move the TF NER example (#10276) 2021-02-19 16:06:13 -05:00
Joe Davison
cbadb5243c Zero shot distillation script cuda patch (#10284) 2021-02-19 14:06:57 -05:00
Joe Davison
c6fe17557e Script for distilling zero-shot classifier to more efficient student (#10244) 
* add zero-shot distillation script

* readme wordsmithing

* clean up code

* add multi-gpu teacher inference
plus tidying up more code

* add use_fast_tokenizer arg

* update results in readme

* more readme wordsmithing

* style

* Add handle to readme

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

* fix code block

* add error+docs about distributed & tpu

* add @sgugger format requests

* xla -> tpu

* support fp16 for teacher preds

* no checkpoint by default

* add demo colab link

* add model sharing prompt + model link

* correct resulting acc of example

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
 2021-02-18 17:08:45 -05:00
Stas Bekman
97e688bc22 [Trainer] memory tracker metrics (#10225) 
* memory tracker metrics

* go back to eval for somewhat consistency

* handle no-gpu case

* deal with stackable eval calls

* restore callback order

* style

* simplify the API

* add test

* docs

* consistently use eval_ prefix

* improve docs

* Update src/transformers/trainer_utils.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* rename method

* style

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2021-02-18 09:27:32 -08:00
Stas Bekman
d1eb88f42d [CI] 2 fixes (#10248) 
* fix invalid port

* missing requirements
 2021-02-17 14:12:39 -08:00
Zhang Cheng
df1b0fb54d set tgt_lang of MBart Tokenizer for summarization (#10205) 2021-02-16 09:39:37 -05:00
Suraj Patil
1c8c2d9ab3 [WIP][examples/seq2seq] move old s2s scripts to legacy (#10136) 
* move old s2s scripts to legacy

* add the tests back

* proper rename

* restore

* Apply suggestions from code review

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

Co-authored-by: Stas Bekman <stas@stason.org>
Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2021-02-15 10:48:02 -08:00
Stas Bekman
0b1f552a24 fix run_seq2seq.py; porting trainer tests to it (#10162) 
* fix run_seq2seq.py; porting DeepSpeed tests to it

* unrefactor

* defensive programming

* defensive programming 2

* port the rest of the trainer tests

* style

* a cleaner scripts dir finder

* cleanup
 2021-02-15 09:12:17 -08:00
Suraj Patil
f51188cbe7 [examples/run_s2s] remove task_specific_params and update rouge computation (#10133) 
* fix rouge metrics and task specific params

* fix typo

* round metrics

* typo

* remove task_specific_params
 2021-02-12 17:18:21 +05:30
Stas Bekman
b54cb0bd82 [DeepSpeed in notebooks] Jupyter + Colab (#10130) 
* init devices/setup explicitly

* docs + test

* simplify

* cleanup

* cleanup

* cleanup

* correct the required dist setup

* derive local_rank from env LOCAL_RANK
 2021-02-11 14:02:05 -08:00
Qbiwan
8dcfaea08d Update run_xnli.py to use Datasets library (#9829) 
* remove xnli_compute_metrics, add load_dataset, load_metric, set_seed,metric.compute,load_metric

* fix

* fix

* fix

* push

* fix

* everything works

* fix init

* fix

* special treatment for sepconv1d

* style

* 🙏🏽

* add doc and cleanup


* fix doc

* fix doc again

* fix doc again

* Apply suggestions from code review

* make style

* Proposal that should work

* Remove needless code

* Fix test

* Apply suggestions from code review

* remove xnli_compute_metrics, add load_dataset, load_metric, set_seed,metric.compute,load_metric

* amend README

* removed data_args.task_name and replaced with task_name = "xnli"; use split function to load train and validation dataset separately; remove __post_init__; remove flag --task_name from README.

* removed dict task_to_keys, use str "xnli" instead of variable task_name, change preprocess_function to use examples["premise"], examples["hypothesis"] directly, remove sentence1_key and sentence2_key, change compute_metrics function to cater only to accuracy metric, add condition for train_langauge is None when using dataset.load_dataset()

* removed `torch.distributed.barrier()` and `import torch` as `from_pretrained` is able to do the work; amend README
 2021-02-11 10:27:23 +05:30
Stas Bekman
77b862847b [DeepSpeed] restore memory for evaluation (#10114) 
* free up memory at the end of train

* rework tests

* consistent formatting

* correction
 2021-02-10 09:09:48 -08:00
Lysandre Debut
0d8e554d42 Line endings should be LF across repo and not CRLF (#10119) 2021-02-10 10:50:00 -05:00
Boris Dayma
7c7962ba89 doc: update W&B related doc (#10086) 
* doc: update W&B related doc

* doc(wandb): mention report_to

* doc(wandb): commit suggestion

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* doc(wandb): fix typo

* doc(wandb): remove WANDB_DISABLED

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2021-02-09 14:47:52 -05:00
Suraj Patil
63fddcf69c [examples/s2s] add test set predictions (#10085) 
* add do_predict, pass eval_beams durig eval

* update help

* apply suggestions from code review
 2021-02-09 20:41:41 +05:30
Stas Bekman
781220acab transition to new tests dir (#10080) 2021-02-08 12:41:52 -08:00
Stas Bekman
322037e842 [trainer] deepspeed bug fixes and tests (#10039) 
* deepspeed bug fixes and tests

* manual wrap?
 2021-02-08 09:44:02 -08:00
Olivier
ece6c51458 [s2s examples] Replace -100 token ids with the tokenizer pad_id for compute_metrics (#10046) 
* replace -100 token ids with the tokenizer pad_id for compute_metrics

* fixed typo for label_ids
 2021-02-08 10:08:16 -05:00
Sylvain Gugger
b01483faa0 Truncate max length if needed in all examples (#10034) 2021-02-08 05:03:55 -05:00
Stas Bekman
24db8cc329 Can't mix --fp16 and --device cpu (#10041) 2021-02-07 17:54:20 -08:00
Stas Bekman
769948fad2 json to jsonlines, and doc, and typo (#10043) 2021-02-07 17:51:34 -08:00
Stas Bekman
8ea412a86f [examples] make run scripts executable (#10037) 
* make executable

* make executable

* same for the template

* cleanup
 2021-02-05 15:51:18 -08:00
Suraj Patil
1cd16512dc [examples/seq2seq] support label smoothing (#9844) 
* add prepare_decoder_input_ids_from_labels in s2s models

* support lbl smoothing and enc/emb freezing

* fix freezing

* use pad_token_id from config

* remove embed freezing and add warning

* prepare decoder_input_ids inside DataCollatorForSeq2Seq
 2021-02-05 23:21:57 +05:30
Suraj Patil
bca0dd5ee3 [run_clm.py] fix getting extention 2021-02-03 20:14:42 +05:30
Stas Bekman
d55e10beab [research proj] [lxmert] rm bleach dependency (#9970) 
Looks like a vulnerability and it's not really used anywhere in the code, so just as well remove it completely from deps.
https://github.com/huggingface/transformers/security/dependabot/examples/research_projects/lxmert/requirements.txt/bleach/open
 2021-02-03 05:24:40 -05:00
Patrick von Platen
538b3b4607 [Tokenizer Utils Base] Make pad function more flexible (#9928) 
* change tokenizer requirement

* split line

* Correct typo from list to str

* improve style

* make other function pretty as well

* add comment

* correct typo

* add new test

* pass tests for tok without padding token

* Apply suggestions from code review
 2021-02-02 10:35:27 +03:00
Sylvain Gugger
115d97dd2f Remove subclass for sortish sampler (#9907) 
* Remove subclass for sortish sampler

* Use old Seq2SeqTrainer in script

* Styling
 2021-02-01 08:06:32 -05:00
wlhgtc
1682804ebd Fit chinese wwm to new datasets (#9887) 
* MOD: fit chinese wwm to new datasets

* MOD: move wwm to new folder

* MOD: formate code

* Styling

* MOD add param and recover trainer

Co-authored-by: Sylvain Gugger <sylvain.gugger@gmail.com>
 2021-02-01 03:37:59 -05:00
Stas Bekman
6bab83683b fix logger format for non-main process (#9911) 2021-02-01 03:08:12 -05:00
Stas Bekman
6bf94bc0b6 correctly handle mt5 (#9879) 2021-01-29 08:11:22 -08:00
Sylvain Gugger
b4e559cfa1 Deprecate model_path in Trainer.train (#9854) 2021-01-28 08:32:46 -05:00
Sylvain Gugger
f2fabedbab Setup logging with a stdout handler (#9816) 2021-01-27 03:39:11 -05:00
Yusuke Mori
059bb25817 Fix a bug in run_glue.py (#9812) (#9815) 2021-01-26 14:32:19 -05:00
Magdalena Biesialska
8f6c12d306 Fix fine-tuning translation scripts (#9809) 2021-01-26 11:30:31 -05:00
Andrea Cappelli
10e5f28212 Improve pytorch examples for fp16 (#9796) 
* Pad to 8x for fp16 multiple choice example (#9752)

* Pad to 8x for fp16 squad trainer example (#9752)

* Pad to 8x for fp16 ner example (#9752)

* Pad to 8x for fp16 swag example (#9752)

* Pad to 8x for fp16 qa beam search example (#9752)

* Pad to 8x for fp16 qa example (#9752)

* Pad to 8x for fp16 seq2seq example (#9752)

* Pad to 8x for fp16 glue example (#9752)

* Pad to 8x for fp16 new ner example (#9752)

* update script template #9752

* Update examples/multiple-choice/run_swag.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update examples/question-answering/run_qa.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update examples/question-answering/run_qa_beam_search.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* improve code quality #9752

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2021-01-26 04:47:07 -05:00
Sylvain Gugger
caf4abf768 Auto-resume training from checkpoint (#9776) 
* Auto-resume training from checkpoint

* Update examples/text-classification/run_glue.py

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

* Roll out to other examples

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
 2021-01-25 12:03:51 -05:00
Wilfried L. Bounsi
9152f16023 Fix broken [Open in Colab] links (#9761) 2021-01-23 15:11:46 +05:30
Sylvain Gugger
411c582109 Fixes to run_seq2seq and instructions (#9734) 
* Fixes to run_seq2seq and instructions

* Add more defaults for summarization
 2021-01-22 10:03:57 -05:00
Stefan Schweter
08b22722c7 examples: fix XNLI url (#9741) 2021-01-22 18:13:52 +05:30
Sylvain Gugger
5f80c15ef5 Fix memory regression in Seq2Seq example (#9713) 
* Fix memory regression in Seq2Seq example

* Fix test and properly deal with -100

* Easier condition with device safety

* Patch for MBartTokenzierFast
 2021-01-21 12:05:46 -05:00
Sylvain Gugger
582f516adb Use datasets squad_v2 metric in run_qa (#9677) 2021-01-20 04:52:13 -05:00
Sylvain Gugger
a1ad16a446 Restrain tokenizer.model_max_length default (#9681) 
* Restrain tokenizer.model_max_length default

* Fix indent
 2021-01-20 04:17:39 -05:00
Sylvain Gugger
e4c06ed664 New run_seq2seq script (#9605) 
* New run_seq2seq script

* Add tests

* Mark as slow

* Update examples/seq2seq/run_seq2seq.py

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

* Update src/transformers/data/data_collator.py

Co-authored-by: Suraj Patil <surajp815@gmail.com>

* Update src/transformers/data/data_collator.py

Co-authored-by: Suraj Patil <surajp815@gmail.com>

* Address review comments

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: Suraj Patil <surajp815@gmail.com>
 2021-01-19 15:22:17 -05:00
Sylvain Gugger
97b787fb4e Fix old Seq2SeqTrainer (#9675) 2021-01-19 09:56:25 -05:00