SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity
7,233 Commits 5 Branches 239 Tags
f086652b16e59bece9571fb9a266557ad3181b2a
Commit Graph

45 Commits

Author SHA1 Message Date
Sylvain Gugger
f086652b16 Add option to log only once in multinode training (#11819) 
* Add option to long only once in multinode training

* Use an alternate property
 2021-05-25 08:03:43 -04:00
Bhadresh Savani
1d30ec95c7 [Examples] Fixes inconsistency around eval vs val and predict vs test (#11380) 
* added changes for uniformity

* modified files

* corrected typo

* fixed qa scripts

* fix typos

* fixed predict typo in qa no trainer

* fixed test file

* reverted trainer changes

* reverted trainer changes in custom exmaples

* updated readme

* added changes in deepspeed test

* added changes for predict and eval
 2021-04-26 09:24:31 -07:00
Sylvain Gugger
acc3bd9d2a Enforce string-formatting with f-strings (#10980) 
* First third

* Styling and fix mistake

* Quality

* All the rest

* Treat %s and %d

* typo

* Missing )

* Apply suggestions from code review

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
 2021-03-31 10:00:27 -04:00
Bhadresh Savani
7ef40120a0 [Examples] Added predict stage and Updated Example Template (#10868) 
* added predict stage

* added test keyword in exception message

* removed example specific saving predictions

* fixed f-string error

* removed extra line

Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>

Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>
 2021-03-23 10:37:59 -07:00
Sylvain Gugger
bf1f43fbd7 Update the example template for a no Trainer option (#10865) 2021-03-23 10:02:39 -04:00
Bhadresh Savani
ac17f71159 added max_sample args and metrics changes (#10602) 2021-03-09 12:06:56 -05:00
Stas Bekman
8ea412a86f [examples] make run scripts executable (#10037) 
* make executable

* make executable

* same for the template

* cleanup
 2021-02-05 15:51:18 -08:00
Sylvain Gugger
b4e559cfa1 Deprecate model_path in Trainer.train (#9854) 2021-01-28 08:32:46 -05:00
Sylvain Gugger
f2fabedbab Setup logging with a stdout handler (#9816) 2021-01-27 03:39:11 -05:00
Andrea Cappelli
10e5f28212 Improve pytorch examples for fp16 (#9796) 
* Pad to 8x for fp16 multiple choice example (#9752)

* Pad to 8x for fp16 squad trainer example (#9752)

* Pad to 8x for fp16 ner example (#9752)

* Pad to 8x for fp16 swag example (#9752)

* Pad to 8x for fp16 qa beam search example (#9752)

* Pad to 8x for fp16 qa example (#9752)

* Pad to 8x for fp16 seq2seq example (#9752)

* Pad to 8x for fp16 glue example (#9752)

* Pad to 8x for fp16 new ner example (#9752)

* update script template #9752

* Update examples/multiple-choice/run_swag.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update examples/question-answering/run_qa.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update examples/question-answering/run_qa_beam_search.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* improve code quality #9752

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2021-01-26 04:47:07 -05:00
Sylvain Gugger
caf4abf768 Auto-resume training from checkpoint (#9776) 
* Auto-resume training from checkpoint

* Update examples/text-classification/run_glue.py

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

* Roll out to other examples

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
 2021-01-25 12:03:51 -05:00
Sylvain Gugger
453a70d4cb Allow example to use a revision and work with private models (#9407) 
* Allow example to use a revision and work with private models

* Copy to other examples and template

* Styling
 2021-01-06 06:49:23 -05:00
Sylvain Gugger
ab17758874 Add speed metrics to all example scripts + template (#9260) 2020-12-22 14:02:26 -05:00
Sylvain Gugger
00aa9dbca2 Copyright (#8970) 
* Add copyright everywhere missing

* Style
 2020-12-07 18:36:34 -05:00
Sylvain Gugger
a0c62d2493 Fix training from scratch in new scripts (#8623) 2020-11-18 12:15:26 -05:00
Julien Chaumond
042a6aa777 Tokenizers: ability to load from model subfolder (#8586) 
* <small>tiny typo</small>

* Tokenizers: ability to load from model subfolder

* use subfolder for local files as well

* Uniformize model shortcut name => model id

* from s3 => from huggingface.co

Co-authored-by: Quentin Lhoest <lhoest.q@gmail.com>
 2020-11-17 08:58:45 -05:00
Sylvain Gugger
cf89724696 Fix validation file loading in scripts (#8298) 2020-11-04 10:42:18 -05:00
Sylvain Gugger
cdc48ce92d Finalize lm examples (#8188) 
* Finish the cleanup of the language-modeling examples

* Update main README

* Apply suggestions from code review

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

* Apply suggestions from code review

Co-authored-by: Thomas Wolf <thomwolf@users.noreply.github.com>

* Propagate changes

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
Co-authored-by: Thomas Wolf <thomwolf@users.noreply.github.com>
 2020-10-30 14:20:18 -04:00
Sylvain Gugger
691176283d Add a template for examples and apply it for mlm and plm examples (#8153) 
* Add a template for example scripts and apply it to mlm

* Formatting

* Fix test

* Add plm script

* Add a template for example scripts and apply it to mlm

* Formatting

* Fix test

* Add plm script

* Add a template for example scripts and apply it to mlm

* Formatting

* Fix test

* Add plm script

* Styling
 2020-10-29 13:38:11 -04:00
Santiago Castro
969859d5f6 Fix doc errors and typos across the board (#8139) 
* Fix doc errors and typos across the board

* Fix a typo

* Fix the CI

* Fix more typos

* Fix CI

* More fixes

* Fix CI

* More fixes

* More fixes
 2020-10-29 10:33:33 -04:00
Stas Bekman
b0cbcdb05b [logging] remove no longer needed verbosity override (#7100) 2020-09-15 04:01:14 -04:00
Lysandre
a75c64d80c Black 20 release 2020-08-26 17:20:22 +02:00
Teven
5a0dac53bf Empty assert hunt (#6056) 
* Fixed empty asserts

* black-reformatted stragglers in templates

* More code quality checks

* Update src/transformers/convert_marian_to_pytorch.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update src/transformers/convert_marian_to_pytorch.py

Co-authored-by: Sam Shleifer <sshleifer@gmail.com>

* removed unused line as per @sshleifer

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Sam Shleifer <sshleifer@gmail.com>
 2020-08-03 10:19:03 +02:00
Sam Shleifer
feeb956a19 [docs] Add integration test example to copy pasta template (#5961) 
Co-authored-by: Julien Chaumond <chaumond@gmail.com>
 2020-07-22 12:48:38 -04:00
Sam Shleifer
45e26125de save_pretrained: mkdir(exist_ok=True) (#5258) 
* all save_pretrained methods mkdir if not os.path.exists
 2020-06-28 14:53:47 -04:00
Julien Chaumond
d4c2cb402d Kill model archive maps (#4636) 
* Kill model archive maps

* Fixup

* Also kill model_archive_map for MaskedBertPreTrainedModel

* Unhook config_archive_map

* Tokenizers: align with model id changes

* make style && make quality

* Fix CI
 2020-06-02 09:39:33 -04:00
Julien Chaumond
a8e3336a85 [examples] Use AutoModels in more examples 2020-03-23 20:11:14 -04:00
Victor SANH
6b1ff25084 fix n_gpu count when no_cuda flag is activated (#3077) 
* fix n_gpu count when no_cuda flag is activated

* someone was left behind
 2020-03-02 10:20:21 -05:00
Julien Plu
ca1d66734d Apply quality and style requirements once again 2020-01-29 12:06:13 -05:00
Julien Plu
0731fa1587 Apply quality and style requirements 2020-01-29 12:06:13 -05:00
Julien Plu
7fc628d98e Apply style 2020-01-29 11:47:49 -05:00
Julien Plu
64ca855617 Add TF2 XLM-RoBERTa model 2020-01-29 11:47:49 -05:00
alberduris
81d6841b4b GPU text generation: mMoved the encoded_prompt to correct device 2020-01-06 15:11:12 +01:00
alberduris
dd4df80f0b Moved the encoded_prompts to correct device 2020-01-06 15:11:12 +01:00
Aymeric Augustin
495580dad1 Remove unused variables in templates. 2019-12-23 22:38:18 +01:00
Aymeric Augustin
1c62e87b34 Use built-in open(). 
On Python 3, `open is io.open`.
 2019-12-22 18:38:56 +01:00
Aymeric Augustin
d6eaf4e6d2 Update comments mentioning Python 2. 2019-12-22 18:38:56 +01:00
Aymeric Augustin
c824d15aa1 Remove __future__ imports. 2019-12-22 17:47:54 +01:00
Aymeric Augustin
fa2ccbc081 Fix E266 flake8 warning (x90). 2019-12-22 10:59:08 +01:00
Aymeric Augustin
631be27078 Fix E722 flake8 warnings (x26). 2019-12-22 10:59:07 +01:00
Aymeric Augustin
158e82e061 Sort imports with isort. 
This is the result of:

    $ isort --recursive examples templates transformers utils hubconf.py setup.py
 2019-12-22 10:57:46 +01:00
Aymeric Augustin
fa84ae26d6 Reformat source code with black. 
This is the result of:

    $ black --line-length 119 examples templates transformers utils hubconf.py setup.py

There's a lot of fairly long lines in the project. As a consequence, I'm
picking the longest widely accepted line length, 119 characters.

This is also Thomas' preference, because it allows for explicit variable
names, to make the code easier to understand.
 2019-12-21 17:52:29 +01:00
Rémi Louf
2276bf69b7 update the examples, docs and template 2019-11-14 20:38:02 +01:00
thomwolf
89d6272898 Fix #1623 2019-11-04 16:21:12 +01:00
thomwolf
7f4226f9e6 adding templates 2019-10-30 11:31:56 +01:00