SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity
8,761 Commits 5 Branches 239 Tags
e57468b8a85bad5cc17efbfcfdd3eecb9b8a62ec
Commit Graph

1900 Commits

Author SHA1 Message Date
Anton Lozhkov
e57468b8a8 Update README.md (#15239) 
Add an OVHcloud tutorial URL for the Robust Speech Challenge
 2022-01-20 11:46:50 +03:00
Patrick von Platen
691878ee2f Update README.md (#15233) 2022-01-19 18:03:17 +01:00
Suraj Patil
2a5a384970 fix speech event readme (#15227) 2022-01-19 15:30:03 +01:00
Patrick von Platen
6d92c429c7 Update README.md (#15226) 2022-01-19 15:23:00 +01:00
Patrick von Platen
19c217b4b7 Update README.md 2022-01-19 15:21:03 +01:00
Patrick von Platen
5439cda7f0 Update README.md 2022-01-19 15:19:57 +01:00
Kamal Raj
d1f5ca1afd [FLAX] glue training example refactor (#13815) 
* refactor run_flax_glue.py

* updated readme

* rm unused import and args typo fix

* refactor

* make consistent arg name across task

* has_tensorboard check

* argparse -> argument dataclasses

* refactor according to review

* fix
 2022-01-19 12:04:51 +01:00
Patrick von Platen
e118e085ea [Robust Speech Event] Add guides (#15155) 
* up

* improve readme

* up

* up

* more info

* up

* up

* Apply suggestions from code review

Co-authored-by: Anton Lozhkov <aglozhkov@gmail.com>

* add more stuff for eval

* update

* up

* Update README.md

* Update examples/research_projects/xls_r/README.md

Co-authored-by: Omar Sanseviero <osanseviero@users.noreply.github.com>

* apply omar's suggestions

Co-authored-by: Anton Lozhkov <aglozhkov@gmail.com>
Co-authored-by: Omar Sanseviero <osanseviero@users.noreply.github.com>
 2022-01-18 18:44:48 +01:00
Sylvain Gugger
6f0a9b41ef Remove dependency to quiet Dependabot (#15205) 2022-01-18 09:44:35 -05:00
Sylvain Gugger
531336bbfd Fix deprecation warnings for int div (#15180) 
* Fix deprecation warnings for int div

Co-authored-by: mgoldey <matthew.goldey@gmail.com>

* Fix import

* ensure that tensor output is python scalar

* make backward compatible

* make code more readable

* adapt test functions

Co-authored-by: mgoldey <matthew.goldey@gmail.com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
 2022-01-18 07:28:53 -05:00
Sylvain Gugger
96881729ce Remove assert on optional arg 2022-01-13 17:34:41 -05:00
Stas Bekman
762416ffa8 [examples/flax/language-modeling] set loglevel (#15129) 2022-01-13 15:17:28 +01:00
Edoardo Federici
9a94bb8e21 mBART support for run_summarization.py (#15125) 
* Update run_summarization.py

* Fixed languages and added missing code

* fixed obj, docs, removed source_lang and target_lang

* make style, run_summarization.py reformatted
 2022-01-12 16:39:33 -05:00
Leandro von Werra
aa0135f2e0 fix: switch from slow to generic tokenizer class (#15122) 2022-01-12 09:12:43 -05:00
Russell Klopfer
27b819b0e3 use block_size instead of max_seq_length in tf run_clm example (#15036) 
* use block_size instead of max_seq_length

* fixup

* remove pad_to_block_size

Co-authored-by: Russell Klopfer <russell@kloper.us>
 2022-01-12 08:57:00 -05:00
Patrick von Platen
d72343d2b8 [Wav2Vec2 Speech Event] Add speech event v2 (#15083) 
* up

* up

* up

* up

* up

* up

* improve

* up

* up

* Update src/transformers/trainer.py

* up

* up

* up
 2022-01-10 10:46:21 +01:00
flozi00
b67f345d00 Update run_speech_recognition_seq2seq.py (#14967) 2022-01-06 19:26:45 +03:00
Yih-Dar
9f89fa02ed Add Flax image captioning example (#14864) 
* add image captioning example

* update README

* fix style & quality

* simplify

* apply review suggestions

* Apply suggestions from code review

Co-authored-by: Suraj Patil <surajp815@gmail.com>

* Apply suggestions from code review

Co-authored-by: Suraj Patil <surajp815@gmail.com>

* Apply review suggestions

* add comments about using np instead jax array

* remove unused lines

* add model creation script

* only support from_pretrained

* fix style

* fix

* not use cache_dir when creating model

* fix tokenizer creation

* update README

* fix quality

* apply suggestion

* simplify some blocks

* Update examples/flax/image-captioning/README.md


* Update examples/flax/image-captioning/run_image_captioning_flax.py

Co-authored-by: Suraj Patil <surajp815@gmail.com>

* apply suggestion

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
Co-authored-by: Suraj Patil <surajp815@gmail.com>
 2022-01-06 14:00:54 +01:00
flozi00
774ed4a027 Fix Code block (#14983) 2022-01-04 12:59:20 +01:00
Patrick von Platen
600496fa50 [Wav2Vec2] Rename model's feature extractor to feature encoder (#14959) 
* rename classes

* clean up more namings

* remove bogus file

* Apply suggestions from code review

* Apply suggestions from code review

* replace more names

* more regex replace

* make style

* correct

* correct more

* make style

* finish

* correct more in wav2vec2

* make style

* improve freeze_extractor

* add aliases

* add tf aliases
 2021-12-28 20:33:23 +01:00
Patrick von Platen
f80775df2b Update README.md (#14965) 2021-12-28 13:41:27 +01:00
Patrick von Platen
1c121916f3 Add Speech Seq2Seq Training script (#14792) 
* start

* add gradient checkpointing and feature extractor freezing

* Apply suggestions from code review

* up

* up

* up

* correct

* up

* more changes

* up

* up

* up

* remove rst
 2021-12-28 10:20:51 +01:00
Leandro von Werra
1d651868d6 add custom stopping criteria to human eval script (#14897) 2021-12-23 14:59:11 +01:00
lewtun
355dc0ce67 Fix installation instructions for BART ONNX example (#14885) 2021-12-23 04:05:32 -05:00
Patrick von Platen
fa39ff9fc4 Docs for v4.16.0dev0 2021-12-22 20:39:44 +01:00
Patrick von Platen
05fa1a7ac1 Release: v4.15.0
Some checks failed
Release - Conda / build_and_package (push) Has been cancelled
Details
2021-12-22 18:43:15 +01:00
Mario Šaško
1045a36c1f Fix pytorch image classification example (#14883) 
* Update example

* Remove skip in tests
 2021-12-22 14:42:19 +01:00
Sylvain Gugger
e51c7b5872 Skip failing test 2021-12-21 15:15:17 -05:00
Stas Bekman
033c3ed95a [examples/summarization] deal with None in data records (#14816) 
* [examples/summarization] deal with None in data records

* rewrite to use a simpler (slower) variant
 2021-12-21 09:17:28 -08:00
Patrick von Platen
7ae6f07004 [ASR example] Improve example + add more examples (#14848) 
* up

* load up

* up
 2021-12-21 13:12:22 +01:00
Patrick von Platen
c4a96cecbc Wav2Vec2 meets phonemes (#14353) 
* up

* add tokenizer

* improve more

* finish tokenizer

* finish

* adapt speech recognition script

* adapt convert

* more fixes

* more fixes

* update phonemizer wav2vec2

* better naming

* fix more tests

* more fixes swedish

* correct tests

* finish

* improve script

* remove file

* up

* lets get those 100 model architectures until the end of the month

* make fix-copies

* correct more

* correct script

* more fixes

* more fixes

* add to docs

* Apply suggestions from code review

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* replace assert

* fix copies

* fix docs

* new try docs

* boom boom

* update

* add phonemizer to audio tests

* make fix-copies

* up

* upload models

* some changes

* Update tests/test_tokenization_wav2vec2_phoneme.py

Co-authored-by: Anton Lozhkov <aglozhkov@gmail.com>

* more fixes

* remove @

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Anton Lozhkov <aglozhkov@gmail.com>
 2021-12-17 19:56:44 +01:00
Lysandre
7c9c41f43c Docs for v4.14.0 2021-12-15 18:29:53 +01:00
Lysandre
960d8cb41d Release: v4.14.0
Some checks failed
Release - Conda / build_and_package (push) Has been cancelled
Details
2021-12-15 18:20:35 +01:00
Yih-Dar
a94105f95f Fix preprocess_function in run_summarization_flax.py (#14769) 
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2021-12-15 11:36:28 +01:00
Benjamin Minixhofer
2a606f9974 Make data shuffling in run_clm_flax.py respect global seed (#13410) 
* use jax and jnp instead of numpy in data_loader

* return batches as np.ndarray
 2021-12-14 11:04:43 +01:00
Josué Nascimento
971e36667a Change how to load config of XLNetLMHeadModel (#14746) 2021-12-13 12:34:26 -05:00
Nathan Cooper
48bf7e47a0 Code parrot minor fixes/niceties (#14666) 
* Add some nicety flags for better controlling evaluation.

* Fix dependency issue with outdated requirement

* Add additional flag to example to ensure eval is done

* Wrap code into main function for accelerate launcher to find

* Fix valid batch size flag in readme

* Add note to install git-lfs when initializing/training the model

* Update examples/research_projects/codeparrot/scripts/arguments.py

Co-authored-by: Leandro von Werra <lvwerra@users.noreply.github.com>

* Update examples/research_projects/codeparrot/README.md

Co-authored-by: Leandro von Werra <lvwerra@users.noreply.github.com>

* Revert "Wrap code into main function for accelerate launcher to find"

This reverts commit ff11df1c810d4df198d04b827538eb4572147ba3.

* Fix formatting issue

* Move git-lfs instructions to installation section

* Add a quick check before code generation for code evaluation

* Fix styling issue

* Update examples/research_projects/codeparrot/scripts/human_eval.py

Co-authored-by: Leandro von Werra <lvwerra@users.noreply.github.com>

* Make iterable dataset use passed in tokenizer rather than globally defined one

Co-authored-by: Leandro von Werra <lvwerra@users.noreply.github.com>
Co-authored-by: ncoop57 <nac33@students.uwf.edu>
 2021-12-13 09:30:50 +01:00
Suraj Patil
6a025487a6 [Flax examples] remove dependancy on pytorch training args (#14636) 
* use custom training arguments

* update tests
 2021-12-12 09:19:12 +05:30
Lysandre
ab31b3e41b Docs for v4.14.0dev0 2021-12-09 17:09:23 +01:00
Lysandre
4da3a696e4 Release: v4.13.0
Some checks failed
Release - Conda / build_and_package (push) Has been cancelled
Details
2021-12-09 16:55:21 +01:00
Gaurang Tandon
4ea19de80c fix: verify jsonlines file in run_translation (#14660) (#14661) 
* fix: verify jsonl in run_translation (#14660)

* fix(run_translation.py): json/jsonl validation

Both json and jsonl are to be accepted as valid jsonlines file extension

* fix(run_translation.py): make black happy

* Ran make style
 2021-12-08 13:25:30 -05:00
Suraj Patil
75ae287aec fix flax examples tests (#14646) 
* make tensorboard optional

* update test_fetcher for flax examples

* make the tests slow
 2021-12-07 00:34:27 +05:30
Suraj Patil
cbe6026536 fix flax example tests (#14643) 2021-12-06 23:14:37 +05:30
Jay Zhang
1ccc033c56 Update the example of exporting Bart + BeamSearch to ONNX module to resolve comments. (#14310) 
* Update code to resolve comments left in previous PR.

* Add README.md file for this example.

* Update examples/onnx/pytorch/translation/README.md

Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>

* Update examples/onnx/pytorch/translation/README.md

Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>

* Update examples/onnx/pytorch/translation/README.md

Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>

* Update README.md file to resolve comments.

* Add a section name.

* Update examples/onnx/pytorch/translation/README.md

Co-authored-by: Gary Miguel <garymm@garymm.org>

* Add more comments for _convert_past_list_to_tuple().

* Change the default file name to a consistent one.

* Fix a format issue.

* Update examples/onnx/pytorch/translation/README.md

Co-authored-by: Gary Miguel <garymm@garymm.org>

* Update examples/onnx/pytorch/translation/run_onnx_exporter.py

Co-authored-by: Gary Miguel <garymm@garymm.org>

* Update examples/onnx/pytorch/translation/README.md

Co-authored-by: lewtun <lewis.c.tunstall@gmail.com>

* Change the folder to summarization and address some other coments.

* Update the torch version.

Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
Co-authored-by: Gary Miguel <garymm@garymm.org>
Co-authored-by: lewtun <lewis.c.tunstall@gmail.com>
 2021-12-06 14:01:51 +01:00
Julien Chaumond
6cdc3a7844 [urls to hub] Replace outdated model tags with their now-canonical pipeline types (#14617) 
* Replace outdated model tags with their now-canonical pipeline types

* spam the CI till it's green
 2021-12-06 04:35:01 -05:00
Suraj Patil
c5bd732ac6 Add Flax example tests (#14599) 
* add test for glue

* add tests for clm

* fix clm test

* add summrization tests

* more tests

* fix few tests

* add test for t5 mlm

* fix t5 mlm test

* fix tests for multi device

* cleanup

* ci job

* fix metric file name

* make t5 more robust
 2021-12-06 10:48:58 +05:30
Kamal Raj
803a8cd18f updated readme with proper arguments (#14624) 2021-12-05 22:12:51 -05:00
(Bill) Yuchen Lin
3977b58437 fix a typo (#14626) 2021-12-05 11:31:23 +05:30
Leandro von Werra
43f953cc2e Add CodeParrot 🦜 codebase (#14536) 
* add readme skeleton

* update readme

* add initialization script

* add deduplication script

* add codeparrot training script

* add code generation evaluation

* add validation loss script

* add requirements

* update readme

* tweak readme

* make style

* add highlights to readme

* add CLIs to scripts

* add tokenizer training script

* add docstring to constant length dataset

* fix defaults in arguments

* update readme with cli

* move image to hub

* tweaks of readme

* fix cli commands

* add author

* explain env variables

* fix formatting

* Update examples/research_projects/codeparrot/README.md

Co-authored-by: lewtun <lewis.c.tunstall@gmail.com>

* Apply suggestions from code review

Co-authored-by: lewtun <lewis.c.tunstall@gmail.com>

* replace generic with gpt2 tokenizer

Co-authored-by: lewtun <lewis.c.tunstall@gmail.com>
 2021-12-02 10:41:35 +01:00
Sylvain Gugger
4df7d05a87 Doc new front (#14590) 
* Convert PretrainedConfig doc to Markdown

* Use syntax

* Add necessary doc files (#14496)

* Doc fixes (#14499)

* Fixes for the new front

* Convert DETR file for table

* Title is needed

* Simplify a bit

* Even simpler

* Remove imports

* Fix typo in toctree (#14516)

* Fix checkpoints badge

* Update versions.yml format (#14517)

* Doc new front github actions (#14512)

* Doc new front github actions

* Fix docstring

* Fix feature extraction utils import (#14515)

* Address Julien's comments

* Push to doc-builder

* Ready for merge

* Remove old build and deploy

* Doc misc fixes (#14583)

* Rm versions.yml from doc

* Fix converting.rst

* Rm pretrained_models from toctree

* Fix index links (#14567)

* Fix links in README

* Localized READMEs

* Fix copy script

* Fix find doc script

* Update README_ko.md

Co-authored-by: Julien Chaumond <julien@huggingface.co>

Co-authored-by: Julien Chaumond <julien@huggingface.co>

* Adapt build command to new CLI tools (#14578)

* Fix typo

* Fix doc interlinks (#14589)

* Convert PretrainedConfig doc to Markdown

* Use syntax

* Rm pattern <[a-z]+(.html).*>

* Rm huggingface.co/transformers/master

* Rm .html

* Rm .html from index.mdx

* Rm .html from model_summary.rst

* Update index.mdx rm html

* Update remove .html

* Fix inner doc links

* Fix interlink in preprocssing.rst

* Update pr_checks

Co-authored-by: Sylvain Gugger <sylvain.gugger@gmail.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Convert PretrainedConfig doc to Markdown

* Use syntax

* Add necessary doc files (#14496)

* Doc fixes (#14499)

* Fixes for the new front

* Convert DETR file for table

* Title is needed

* Simplify a bit

* Even simpler

* Remove imports

* Fix checkpoints badge

* Fix typo in toctree (#14516)

* Update versions.yml format (#14517)

* Doc new front github actions (#14512)

* Doc new front github actions

* Fix docstring

* Fix feature extraction utils import (#14515)

* Address Julien's comments

* Push to doc-builder

* Ready for merge

* Remove old build and deploy

* Doc misc fixes (#14583)

* Rm versions.yml from doc

* Fix converting.rst

* Rm pretrained_models from toctree

* Fix index links (#14567)

* Fix links in README

* Localized READMEs

* Fix copy script

* Fix find doc script

* Update README_ko.md

Co-authored-by: Julien Chaumond <julien@huggingface.co>

Co-authored-by: Julien Chaumond <julien@huggingface.co>

* Adapt build command to new CLI tools (#14578)

* Fix typo

* Fix doc interlinks (#14589)

* Convert PretrainedConfig doc to Markdown

* Use syntax

* Rm pattern <[a-z]+(.html).*>

* Rm huggingface.co/transformers/master

* Rm .html

* Rm .html from index.mdx

* Rm .html from model_summary.rst

* Update index.mdx rm html

* Update remove .html

* Fix inner doc links

* Fix interlink in preprocssing.rst

* Update pr_checks

Co-authored-by: Sylvain Gugger <sylvain.gugger@gmail.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Styling

Co-authored-by: Mishig Davaadorj <mishig.davaadorj@coloradocollege.edu>
Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
Co-authored-by: Julien Chaumond <julien@huggingface.co>
 2021-12-01 14:13:02 -05:00