SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity
7,433 Commits 5 Branches 239 Tags
9490d668d2f59ad2e7a4db3dc7ed2f9684af369c
Commit Graph

1656 Commits

Author SHA1 Message Date
Taha ValizadehAslani
9490d668d2 Update run_mlm.py (#12344) 
Before the code could not be used for validation only because of this line:
extension = data_args.train_file.split(".")[-1]
was assuming that extension must be extracted from the training dataset. This line would run regardless of the training or validation options of the user. This would lead to an error if the user only wants to run an evaluation only and does not want to do train (because the training file does not exist). I modified it to extract extension from the training file if the user wants to do train and extract it from the validation file if the user wants to run eval. This way the code can be used for both training and validation separately.
 2021-06-28 07:49:22 -04:00
Bhadresh Savani
ff5cdc086b replace print with logger (#12368) 2021-06-26 09:31:25 -07:00
Bhadresh Savani
539ee456d4 [Examples] Replicates the new --log_level feature to all trainer-based pytorch (#12359) 
* added log_level

* fix comment

* fixed log_level

* Trigger CI

* Unfied logging

* simplified args for log_level
 2021-06-25 14:58:42 -07:00
Stas Bekman
64e6098094 [trainer] add main_process_first context manager (#12351) 
* main_process_first context manager

* handle multi-node, add context description

* sync desc
 2021-06-25 14:58:03 -07:00
Stas Bekman
4a872caef4 remove extra white space from log format (#12360) 2021-06-25 13:20:14 -07:00
Vasudev Gupta
332a245861 Add FlaxBigBird QuestionAnswering script (#12233) 
* port bigbird script

* adapt script a bit

* change location

* adapt more

* save progress

* init commit

* style

* dataset script tested

* readme add
 2021-06-25 18:05:48 +01:00
michal pitr
d4ce31e839 fixed typo (#12356) 2021-06-25 07:49:29 -04:00
Patrick von Platen
aa550c4a11 Update README.md 2021-06-25 11:55:51 +01:00
Marc van Zee
f2c4ce7e33 Add flax/jax quickstart (#12342) 2021-06-24 17:04:18 +01:00
Suraj Patil
aef3823e1a [examples/Flax] move the examples table up (#12341) 2021-06-24 16:03:37 +05:30
Sylvain Gugger
2150dfed31 v4.9.0.dev0 2021-06-23 13:31:19 -04:00
Sylvain Gugger
9252a5127f Release: v4.8.0 2021-06-23 13:25:56 -04:00
Patrick von Platen
44739c8180 [Flax/JAX] Add how to propose projects markdown (#12311) 
* fix_torch_device_generate_test

* remove @

* finish

* make style
 2021-06-23 14:50:35 +01:00
Suraj Patil
c0fe3c9a7a Flax summarization script (#12230) 
* add summrization script

* fix arguments, preprocessing, metrics

* add generation and metrics

* auto model, prediction loop

* prettify

* label smoothing

* adress Sylvain and Patricks suggestions

* dynamically import shift_tokens_right

* fix shift_tokens_right_fn call
 2021-06-23 15:49:30 +05:30
Stas Bekman
ebe5413589 [trainer] 2 bug fixes and a rename (#12309) 
* bug fixes and a rename

* add extended DDP test
 2021-06-22 11:13:23 -07:00
Patrick von Platen
64029abe4c [Flax] Main doc for event orga (#12305) 
* fix_torch_device_generate_test

* remove @

* push

* finish

* some typos

* add more info on communication

* add suggestions
 2021-06-22 18:02:52 +01:00
Stas Bekman
dad414d5f9 [trainer + examples] set log level from CLI (#12276) 
* set log level from CLI

* add log_level_replica + test + extended docs

* cleanup

* Apply suggestions from code review

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* rename datasets objects to allow datasets module

* improve the doc

* style

* doc improve

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2021-06-21 19:30:50 -07:00
Matt
e3cb7a0b60 Tensorflow QA example (#12252) 
* New Tensorflow QA example!

* Style pass

* Updating README.md for the new example

* flake8 fixes

* Update examples/tensorflow/question-answering/README.md

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2021-06-21 16:37:28 +01:00
Vishal Burman
b53bc55ba9 Fix for making student ProphetNet for Seq2Seq Distillation (#12130) 
* make_student.py: fix to make student ProphetNet

* reformat
 2021-06-21 09:36:44 -04:00
Bhavitvya Malik
e43e11260f update desc for map in all examples (#12226) 
* update desc for map in all examples

* added plm

* suggestions
 2021-06-17 15:37:31 -04:00
Lysandre
0daadc1919 Docs for v4.8.0 2021-06-17 18:17:42 +02:00
Lysandre
7a6c9fab8e Release: v4.7.0
Some checks failed
Release - Conda / build_and_package (push) Has been cancelled
Details
2021-06-17 17:57:42 +02:00
Sylvain Gugger
7d7ceca396 Model card defaults (#12122) 
* [WIP] Model card defaults

* finetuned_from default value

* Add all mappings to the mapping file

* Be more defensive on finetuned_from arg

* Add default task tag

* Separate tags from tasks

* Edge case for dataset

* Apply suggestions from code review

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
 2021-06-15 16:01:37 -04:00
kumapo
955b2b97a6 Enable add_prefix_space if model_type is roberta or gpt2 (#12116) 2021-06-15 09:33:21 -04:00
Avital Oliver
9b393240a2 Use a released version of optax rather than installing from Git. (#12173) 
Use a released version of optax rather than installing from Git
 2021-06-15 16:42:51 +05:30
Stas Bekman
88e84186e5 [style] consistent nn. and nn.functional: part 4 examples (#12156) 
* consistent nn. and nn.functional: p4 examples

* restore
 2021-06-14 12:28:24 -07:00
Kumar Abhishek
9de62cfbce [lm examples] Replicate --config_overrides addition to other LM examples (#12135) 
* [lm examples] Replicate --config_overrides addition to other LM examples

* Removing no trainer files changes

* Update README

Co-authored-by: Kumar Abhishek <kabhishek@expedia.com>
 2021-06-14 08:12:22 -04:00
Nicholas Broad
cd7961b632 Use text_column_name variable instead of "text" (#12132) 
* Use text_column_name variable instead of "text"

`text_column_name` was already defined above where I made the changes and it was also used below where I made changes.

This is a very minor change. If a dataset does not use "text" as the column name, then the `tokenize_function` will now use whatever column is assigned to `text_column_name`. `text_column_name` is just the first column name if "text" is not a column name. It makes the function a little more robust, though I would assume that 90% + of datasets use "text" anyway.

* black formatting

* make style

Co-authored-by: Nicholas Broad <nicholas@nmbroad.com>
 2021-06-14 08:11:13 -04:00
Sylvain Gugger
b8ab541340 Don't log anything before logging is setup in examples (#12121) 
* Don't log anything before logging is setup in examples

* Last example
 2021-06-14 08:03:33 -04:00
Patrick von Platen
7566fefa69 [Flax] Add links to google colabs (#12146) 
* fix_torch_device_generate_test

* remove @

* add colab links
 2021-06-14 11:00:29 +01:00
Suraj Patil
d36fce8237 add readme for flax clm (#12111) 
* add readme for flax clm

* use section link for tokenizer

* Apply suggestions from code review

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

* update metrics

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
 2021-06-14 15:03:55 +05:30
Patrick von Platen
16c0efca2c Add mlm pretraining xla torch readme (#12011) 
* fix_torch_device_generate_test

* remove @

* upload

* Apply suggestions from code review

* Apply suggestions from code review

* Apply suggestions from code review

* Update examples/flax/language-modeling/README.md

* add more info

* finish

* fix

Co-authored-by: Patrick von Platen <patrick@huggingface.co>
 2021-06-14 10:31:21 +01:00
Suraj Patil
15b498f3b8 Flax CLM script (#12023) 
* first draft

* max_seq_length => block_size

* fix arg names

* fix typos

* fix loss calculation

* add max examples, fix  train eval steps, metrics

* optimizer mask

* fix perpelexity, metric logging

* fix logging

* data_collator = > data_loader

* refactor loss_fn

* support single GPU

* pass distributed to write_metric

* fix jitting

* fix single device training

* fix single device metrics

* close inner progress bars once finished

* add overwrite_cache arg

* ifx dataset caching issue

* add more logs

* few small fixes,

* address nicholas suggestions

* fix docstr

* address patricks suggestions

* make flake happy

* pass new new_dropout_rng to apply_gradients

* reset train metrics after every epoc

* remove distributed logis, small fixes
 2021-06-11 15:16:20 +05:30
Bhavitvya Malik
d2753dcbec add relevant description to tqdm in examples (#11927) 
* add relevant `desc` in examples

* require_version datasets>=1.8.0
 2021-06-10 15:59:55 -04:00
Matt
bebbdd0fc9 Appending label2id and id2label to models to ensure inference works properly (#12102) 2021-06-10 15:25:04 +01:00
Matt
4cda08decb Minor style edits 2021-06-10 15:10:57 +01:00
Matt
7f08dbd10a Update README.md to cover the TF GLUE example. 2021-06-10 14:33:42 +01:00
Sylvain Gugger
d72e5a3a6d Fix quality 2021-06-10 09:27:11 -04:00
Matt
73a532651a New TF GLUE example (#12028) 
* Pushing partially-complete new GLUE example

* First draft of the new TF GLUE example! Needs a little more testing to be sure but it's almost ready.

* Fix to the fit() call

* Bugfixes, making sure TPU and multi-GPU support is ready

* Remove logger line that depends on Pytorch

* Style pass

* Deleting old TF GLUE example

* Include label2id and id2label in the saved model config

* Don't clobber the existing model.config.label2id

* Style fixes

* Update examples/tensorflow/text-classification/run_glue.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2021-06-10 14:14:37 +01:00
kumapo
472a867626 Add text_column_name and label_column_name to run_ner and run_ner_no_trainer args (#12083) 
* Add text_column_name and label_column_name to run_ner args

* Minor fix: grouping for text and label column name
 2021-06-10 08:03:20 -04:00
Stas Bekman
61e191987d rm require_version_examples (#12088) 2021-06-09 11:02:52 -07:00
Suraj Patil
d1500d9151 pass decay_mask fn to optimizer (#12087) 2021-06-09 18:49:27 +01:00
Anton Lozhkov
d472bd7b18 Wav2Vec2 Pretraining (#11306) 
* Working quantizer forward

* Working quantizer forward

* Clean up unused model parts, test reproducibility

* Working quantizer forward

* Clean up unused model parts, test reproducibility

* Remove custom outputs from the shared ones

* correct conversion

* correct bug

* add first pretrain script

* save intermediate

* static shapes

* save intermediate

* finish first pretrain script version

* more refactor

* remove wanddb

* refactor more

* improve test

* correct perplexity compute bug

* finish model implementation

* add to docs

* finish docs

* finish pretraining script

* finish pretraining script

* remove wandb

* finish PR for merge

* finish config

* finish

* make deepspeed work

* Apply suggestions from code review

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* apply suggestions

* fix flaky test

Co-authored-by: patrickvonplaten <patrick.v.platen@gmail.com>
Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2021-06-09 18:40:56 +01:00
Stas Bekman
d14e0af274 sync LayerDrop for Wav2Vec2Encoder + tests (#12076) 2021-06-09 13:21:03 +01:00
Koichi Yasuoka
82a2b76c95 Update run_ner.py with id2label config (#12001) 2021-06-09 07:27:05 -04:00
Stas Bekman
11d86d3de4 [Deepspeed Wav2vec2] integration (#11638) 
* wip

* wip - but working with https://github.com/microsoft/DeepSpeed/pull/1044

* cleanup

* workaround

* working 5/8 modes

* solve fp32 distributed zero3

* style

* sync

* sync

* rework

* deprecation

* cleanup

* https://github.com/microsoft/DeepSpeed/pull/1044 pr was merged

* clean up

* add a guide

* more prose

* more prose

* fix

* more prose

* sub_group_size was too big

* Apply suggestions from code review

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* refactor

* bug fix

* make the true check explicit

* new deepspeed release

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2021-06-08 12:32:03 -07:00
Sylvain Gugger
fd6902838a Properly indent block_size (#12070) 2021-06-08 10:27:02 -04:00
cdleong
49bee0aea4 Add torch to requirements.txt in language-modeling (#12040) 
* Add torch to requirements.txt in language-modeling

* Update examples/pytorch/language-modeling/requirements.txt

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2021-06-08 09:02:35 -04:00
Mario Šaško
f5eec0d8e9 Replace legacy tensor.Tensor with torch.tensor/torch.empty (#12027) 
* Replace legacy torch.Tensor constructor with torch.{tensor, empty}

* Remove torch.Tensor in examples
 2021-06-08 13:58:38 +01:00
Shamane Siri
e33085d648 updated the original RAG implementation to be compatible with latest Pytorch-Lightning (#11806) 
* updated the original RAG implementation to be compatible with the latest PL version

* updated the requirements.txt file

* execute make style

* code quality test

* code quality

* conflix resolved in requirement.txt

* code quality

* changed the MyDDP class name to CustomDDP
 2021-06-08 13:42:49 +01:00