SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity
7,663 Commits 5 Branches 239 Tags
c164064eefeb180b030c3757b87a4bd3e3d24ac3
Commit Graph

1736 Commits

Author SHA1 Message Date
chutaklee
c164064eef Fix distiller.py (#12910) 
* fix distiller

* fix style
 2021-07-29 02:11:38 +08:00
Sylvain Gugger
3ec851dc5e Fix QA examples for roberta tokenizer (#12928) 2021-07-28 09:47:49 -04:00
Sylvain Gugger
fd85734e0e Add option to set max_len in run_ner (#12929) 2021-07-28 09:38:12 -04:00
Elysium1436
f3d0866ed9 Correct validation_split_percentage argument from int (ex:5) to float (0.05) (#12897) 
* Fixed train_test_split test_size argument

* `Seq2SeqTrainer` set max_length and num_beams only when non None  (#12899)

* set max_length and num_beams only when non None

* fix instance variables

* fix code style

* [FLAX] Minor fixes in CLM example (#12914)

* readme: fix retrieval of vocab size for flax clm example

* examples: fix flax clm example when using training/evaluation files

* Fix module path for symbolic_trace example

Co-authored-by: cchen-dialpad <47165889+cchen-dialpad@users.noreply.github.com>
Co-authored-by: Stefan Schweter <stefan@schweter.it>
Co-authored-by: Sylvain Gugger <sylvain.gugger@gmail.com>
 2021-07-27 21:01:40 -04:00
Stefan Schweter
d3c3e722d6 [FLAX] Minor fixes in CLM example (#12914) 
* readme: fix retrieval of vocab size for flax clm example

* examples: fix flax clm example when using training/evaluation files
 2021-07-27 19:48:04 +05:30
Matt
569f61a760 Add TF multiple choice example (#12865) 
* Add new multiple-choice example, remove old one
 2021-07-26 15:15:51 +01:00
Sylvain Gugger
303989de0e Add accelerate to examples requirements (#12888) 2021-07-26 09:57:34 -04:00
Stas Bekman
98364ea74f [tests] fix logging_steps requirements (#12860) 2021-07-23 08:05:48 -07:00
Lysandre
40de2d5a4f Docs for v4.10.0dev0 2021-07-22 12:52:25 +02:00
Lysandre
72aee83ced Release: v4.9.0
Some checks failed
Release - Conda / build_and_package (push) Has been cancelled
Details
2021-07-22 12:11:55 +02:00
Maxwell Forbes
fcf83011df Fix type of max_seq_length arg in run_swag.py (#12832) 2021-07-22 02:14:14 -04:00
Patrick von Platen
acdd78db08 Update README.md 2021-07-20 16:48:37 +02:00
Patrick von Platen
31d06729f4 Update README.md 2021-07-20 14:19:37 +02:00
Patrick von Platen
13fefdf340 Update README.md 
cc @patil-suraj
 2021-07-20 13:51:15 +02:00
fgaim
66197adc98 Flax MLM: Allow validation split when loading dataset from local file (#12689) 
* Allow validation split when loading dataset from local file

* Flax clm & t5, enable validation split for datasets loaded from local file
 2021-07-20 13:38:25 +02:00
Patrick von Platen
c6b9095cb2 Update README.md 2021-07-17 19:22:26 +02:00
Patrick von Platen
b4b562d834 [Wav2Vec2] Padded vectors should not allowed to be sampled (#12764) 
* fix_torch_device_generate_test

* remove @

* finish

* correct script

* correct script
 2021-07-16 19:07:08 +02:00
Suraj Patil
8ef3f36561 fix typos (#12757) 2021-07-16 16:44:59 +05:30
Patrick von Platen
a76dd7ee82 Update README.md 2021-07-16 00:16:30 +01:00
Patrick von Platen
2e9fb13fb1 [Wav2Vec2] Correctly pad mask indices for PreTraining (#12748) 
* fix_torch_device_generate_test

* remove @

* start adding tests

* correct wav2vec2 pretraining

* up

* up

Co-authored-by: Patrick von Platen <patrick@huggingface.co>
 2021-07-15 21:40:25 +01:00
Suraj Patil
44f5b260fe flax model parallel training (#12590) 
* update scripts

* add copyright

* add logging

* cleanup

* add z loss

* add readme

* shard description

* update readme
 2021-07-14 22:55:44 +05:30
Matt
f9ac677eba Update TF examples README (#12703) 
* Update Transformers README, rename token_classification example to token-classification to be consistent with the others

* Update examples/tensorflow/README.md

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Add README for TF token classification

* Update examples/tensorflow/token-classification/README.md

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update examples/tensorflow/token-classification/README.md

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2021-07-14 15:15:25 +01:00
Patrick von Platen
f4399ec570 Update README.md 2021-07-14 12:54:31 +01:00
Matt
65bf05cd18 Adding TF translation example (#12667) 
* Adding TF translation example

* Fixes and style pass for TF translation example

* Remove unused postprocess_text copied from run_summarization

* Adding README

* Review fixes

* Move changes to model.config to after we've initialized the model
 2021-07-13 19:08:25 +01:00
Nick Doiron
5803a2a7ac Add ByT5 option to example run_t5_mlm_flax.py (#12634) 
* Allow ByT5 type in Flax T5 script

* use T5TokenizerFast

* change up tokenizer config

* model_args

* reorder imports

* Update run_t5_mlm_flax.py
 2021-07-13 13:39:57 +01:00
Omar Sanseviero
c523b241c2 Update timeline for Flax event evaluation 2021-07-12 21:24:58 +02:00
Matt
379f649434 TF summarization example (#12617) 
* Adding a TF summarization example

* Style pass

* Style fixes

* Updates for review comments

* Adding README

* Style pass

* Remove unused import
 2021-07-12 15:58:38 +01:00
Eduardo Gonzalez Ponferrada
2dd9440d08 Point to the right file for hybrid CLIP (#12599) 2021-07-12 12:16:22 +05:30
Bhadresh Savani
de23ecea36 added test file (#12630) 2021-07-12 12:15:14 +05:30
Patrick von Platen
deecdd4939 [Flax] Fix cur step flax examples (#12608) 
* fix_torch_device_generate_test

* remove @

* fix save problem
 2021-07-09 13:51:28 +01:00
Omar Sanseviero
8fe836af5a Add Flax sprint project evaluation section (#12592) 2021-07-09 08:52:30 +02:00
Sylvain Gugger
6f1adc4334 Fix group_lengths for short datasets (#12558) 2021-07-08 07:23:41 -04:00
Ibraheem Moosa
122d7dc34f Remove logging of GPU count etc logging. (#12569) 
Successfully logging this requires Pytorch. For the purposes of this script we are not using Pytorch.
 2021-07-07 23:05:47 +01:00
Suraj Patil
d7e156bd1a fix loading clip vision model (#12566) 2021-07-07 22:50:27 +05:30
Patrick von Platen
7d321b7689 [Flax] Allow retraining from save checkpoint (#12559) 
* fix_torch_device_generate_test

* remove @

* finish
 2021-07-07 19:13:43 +05:30
Souvic Chakraborty
1d6623c6a2 MLM training fails with no validation file(same as #12406 for pytorch now) (#12517) 
* Validation split percentage to be used for custom data files also

Issue same as https://github.com/huggingface/transformers/issues/12406 fixed for pytorch branch run_mlm.py

* Validation split added in the right place

* Update run_clm.py

* validation split added for custom files

* Validation split added for custom files

* Update run_plm.py

* fixed validation split for custom files as input for pytorch examples in lm

* Update run_clm_no_trainer.py

* args modified
 2021-07-07 09:05:44 -04:00
Suraj Patil
2d42915abe [examples/flax] add adafactor optimizer (#12544) 
* add adafactor

* Update examples/flax/language-modeling/run_mlm_flax.py

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
 2021-07-07 11:50:30 +05:30
Patrick von Platen
208df208bf [Flax] Adapt examples to be able to use eval_steps and save_steps (#12543) 
* fix_torch_device_generate_test

* remove @

* up

* up

* correct

* upload

Co-authored-by: Patrick von Platen <patrick@huggingface.co>
 2021-07-06 19:41:51 +01:00
SaulLu
09af5bdea3 Replace nn.Moudle by nn.Module (#12541) 2021-07-06 11:31:45 -04:00
Patrick von Platen
f42a0abf4b Update README.md 2021-07-06 15:14:48 +01:00
Suzana Ilić
029b9d3f40 Update README (#12540) 2021-07-06 16:12:16 +02:00
Suraj Patil
f5b0c1ecf0 [Flax] Fix hybrid clip (#12519) 
* fix saving and loading

* update readme
 2021-07-06 11:12:47 +05:30
Patrick von Platen
7d6285a921 [Wav2Vec2] Flax - Adapt wav2vec2 script (#12520) 
* fix_torch_device_generate_test

* remove @

* adapt flax pretrain script
 2021-07-05 23:49:47 +01:00
Patrick von Platen
4605b2b8ec [Flax] Fix another bug in logging steps (#12516) 
* fix_torch_device_generate_test

* remove @

* up
 2021-07-05 18:35:22 +01:00
Patrick von Platen
d0f7508abe [Flax] Correct logging steps flax (#12515) 
* fix_torch_device_generate_test

* remove @

* push
 2021-07-05 18:21:00 +01:00
Patrick von Platen
bb4ac2b5a8 [Flax] Correct flax training scripts (#12514) 
* fix_torch_device_generate_test

* remove @

* add logging steps

* correct training scripts

* correct readme

* correct
 2021-07-05 18:14:50 +01:00
Matt
ea55675024 NER example for Tensorflow (#12469) 
* NER example for Tensorflow

* Style pass

* Style pass

* Added metric computation on the evaluation set

* Style pass

* Fixed label masking

* Style pass

* Style pass
 2021-07-05 15:42:18 +01:00
Patrick von Platen
9b90810558 [Flax] Dataset streaming example (#12470) 
* fix_torch_device_generate_test

* remove @

* upload

* finish dataset streaming

* adapt readme

* finish

* up

* up

* up

* up

* Apply suggestions from code review

* finish

* make style

* make style2

* finish

Co-authored-by: Patrick von Platen <patrick@huggingface.co>
 2021-07-05 15:13:10 +01:00
Navjot
eceb1042c1 flax.linen.apply takes state as the first param, followed by the input (#12510) 2021-07-05 19:33:14 +05:30
Suraj Patil
f1c81d6b92 [Flax] ViT training example (#12300) 
* begin script

* clean example, add readme

* update readme

* remove decay mask

* remove masking

* update readme & make flake happy
 2021-07-05 18:23:03 +05:30