SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity
11,726 Commits 5 Branches 239 Tags
48d4e147d824efab97637947709d5aa67c809b3d
Commit Graph

11726 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Yih-Dar
48d4e147d8 remove flax file from documentation_tests.txt (#21036) 
remove flax file from `documentation_tests.txt`

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-01-08 12:33:25 +01:00
Sylvain Gugger
d0f324f1e1 Fix warning for MCTC model (#21049) 2023-01-08 10:55:23 +01:00
Sylvain Gugger
9a046cc14e Skip failing test until Athur looks at it. 2023-01-08 04:53:20 -05:00
Arthur
f0577df6de Replace past with past_key_values (#20944) 
* start cleanup

* more updates

* more models are affected

* more updates

* update generation utils

* style

* revert change that removed reorder cachce

* update generation utils

* style

* style

* remove reorder cache
 2023-01-08 10:21:40 +01:00
SABA UL HAQUE
7cb596fa22 fix typo (#21048) 
Typo fix: Corrected the word metada --> metadata
 2023-01-08 10:03:01 +01:00
Kaito Sugimoto
bd9d51263a fix typo (#21042) 2023-01-07 10:13:26 +01:00
Bartosz Szmelczynski
f93c90d217 fix levit timm conversion file (#20938) 
* fix levit timm conversion file

* remove set_defaults
 2023-01-06 13:27:30 +01:00
Ceyda Cinarel
c29bec485e fix parameter name in docstring (#21032) 2023-01-06 07:23:16 -05:00
Dudu Lasry
61e068e5a2 Support turning off the model uploading in ClearML (#20969) 
* Add support for turning off the model uploading in ClearML

* Add documentation for the CLEARML_LOG_MODEL environment variable

* Adjust new doc addition to the new style

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

Co-authored-by: Dudu Lasry <dudu.lasry@viz.ai>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2023-01-06 07:22:19 -05:00
Observer46
ff8dcb5efa Fix arguments passed to predict function in QA Seq2seq training script (#21026) 
fix args passed to predict function
 2023-01-06 07:19:42 -05:00
Roy Hvaara
35a7052b61 [NumPy] Remove references to deprecated NumPy type aliases (#21022) 
[NumPy] Remove references to deprecated NumPy type aliases.

This change replaces references to a number of deprecated NumPy type aliases (np.bool, np.int, np.float, np.complex, np.object, np.str) with their recommended replacement (bool, int, float, complex, object, str).

NumPy 1.24 drops the deprecated aliases, so we must remove uses before updating NumPy.

Co-authored-by: Peter Hawkins <phawkins@google.com>

Co-authored-by: Peter Hawkins <phawkins@google.com>
 2023-01-05 13:02:10 -05:00
Magnus Pierrau
1d21471c78 Added mask_time_prob and mask_time_length arguments to wav2vec2 pretraining script (#20985) 
Added mask_time_prob and mask_time_length arguments to wav2vec2 pretraining script and readme - new branch
 2023-01-05 16:24:55 +00:00
Joao Gante
bc53fc6265 Generate: FLAX uses GenerationConfig as the basis for .generate() parametrization (#21007) 2023-01-05 15:41:37 +00:00
NielsRogge
4f1c9d162e [CLIPSeg] Fix integration test (#20995) 
Fix integration test

Co-authored-by: Niels Rogge <nielsrogge@Nielss-MacBook-Pro.local>
 2023-01-05 14:30:32 +01:00
Sylvain Gugger
12313838d3 Make sure dynamic objects can be saved and reloaded (#21008) 
* Make sure dynamic objects can be saved and reloaded

* Remove processor test
 2023-01-05 07:30:25 -05:00
Younes Belkada
bf82c9b74f [BLIP] Fix daily CI failing test (#20877) 2023-01-05 13:24:31 +01:00
Joao Gante
beb24f2a36 Generate: FLAX infers pad token in its absence and has functional example (#21009) 2023-01-05 11:52:58 +00:00
Joao Gante
480799f718 Generate: post-generate config TF doctest fix (#21018) 2023-01-05 11:38:37 +00:00
Steven Liu
8fb4d0e4b4 Fix callback docstrings (#21005) 
* fix callback docstrings

* format as markdown list

* apply feedback
 2023-01-04 12:59:23 -08:00
dependabot[bot]
b7417bee87 Bump gitpython from 3.0.2 to 3.1.30 in /examples/research_projects/distillation (#21011) 
Bump gitpython in /examples/research_projects/distillation

Bumps [gitpython](https://github.com/gitpython-developers/GitPython) from 3.0.2 to 3.1.30.
- [Release notes](https://github.com/gitpython-developers/GitPython/releases)
- [Changelog](https://github.com/gitpython-developers/GitPython/blob/main/CHANGES)
- [Commits](https://github.com/gitpython-developers/GitPython/compare/3.0.2...3.1.30)

---
updated-dependencies:
- dependency-name: gitpython
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-01-04 15:36:42 -05:00
dependabot[bot]
05b736c16e Bump gitpython from 3.1.18 to 3.1.30 in /examples/research_projects/decision_transformer (#21010) 
Bump gitpython in /examples/research_projects/decision_transformer

Bumps [gitpython](https://github.com/gitpython-developers/GitPython) from 3.1.18 to 3.1.30.
- [Release notes](https://github.com/gitpython-developers/GitPython/releases)
- [Changelog](https://github.com/gitpython-developers/GitPython/blob/main/CHANGES)
- [Commits](https://github.com/gitpython-developers/GitPython/compare/3.1.18...3.1.30)

---
updated-dependencies:
- dependency-name: gitpython
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-01-04 15:36:33 -05:00
Yih-Dar
94db82573e Fix (DeepSpeed) docker image build issue (#21002) 
* Fix docker image build issue

* remove comment

* Add comment

* Update docker/transformers-pytorch-deepspeed-latest-gpu/Dockerfile

Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>
 2023-01-04 21:28:33 +01:00
Joao Gante
b91048968b Generate: Fix CI related to #20727 (#21003) 2023-01-04 20:26:56 +00:00
Sayak Paul
263fd3c4c7 add: task guide on video classification model fine-tuning. (#20827) 
* add: task guide on video classification model fine-tuning.

* apply make style from hf-formatting.

* add: toc entry.

* chore: address PR comments.

Co-authored-by Maria Khalusova

* Reflect Maria's contributions.

Co-authored-by: Maria Khalusova <1065417+MKhalusova@users.noreply.github.com>

* chore: minor correction.

* Apply suggestions from code review

Co-authored-by: Nathan Raw <nxr9266@g.rit.edu>

* PyTorch Video -> PyTorchVideo.

* Apply suggestions from code review

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

* change licensing year.

* minor rewording.

* apply make style.

* address Sylvain's comments.

* replace links.

Co-authored-by: Maria Khalusova <1065417+MKhalusova@users.noreply.github.com>
Co-authored-by: Nathan Raw <nxr9266@g.rit.edu>
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
 2023-01-05 00:43:40 +05:30
Steven Liu
d53f329d88 Update PR template (#21006) 
add maria to pr template
 2023-01-04 11:01:52 -08:00
Sylvain Gugger
7804177af9 Fix repo consistency 2023-01-04 14:00:45 -05:00
Sujay
15e17c99f9 Remove T5 dependency from mT5 model (#20949) 
make mt5 independent from t5
 2023-01-04 13:51:54 -05:00
Steven Liu
9dcc881fa6 Update bug report template (#21004) 
add maria to bug report
 2023-01-04 10:33:15 -08:00
Joao Gante
a6c850e4f4 Generate: TF uses GenerationConfig as the basis for .generate() parametrization (#20994) 2023-01-04 18:23:20 +00:00
milyiyo
3b309818e7 Refactor the function get_results (#20999) 2023-01-04 12:05:36 -05:00
İdil Sülo
926452298d Fix model hub link (#20998) 2023-01-04 12:04:33 -05:00
amyeroberts
56397471b4 Don't call deprecated method (#20904) 2023-01-04 16:59:11 +00:00
Alara Dirik
52c9e6af29 Fix bug in segmentation postprocessing (#20198) 
* Fix post_process_instance_segmentation
* Add test for label fusing
 2023-01-04 18:34:58 +03:00
amyeroberts
292acd71d6 Update image processor parameters if creating with kwargs (#20866) 
* Update parameters if creating with kwargs

* Shallow copy to prevent mutating input

* Pass all args in constructor dict - warnings in init

* Fix typo
 2023-01-04 14:29:48 +00:00
JeongYeon Nam
f9e977be70 auxiliary_loss works for Deformable Detr (#20959) 
fix: auxiliary_loss works

Co-authored-by: Jeongyeon Nam <jy.nam@navercorp.com>
 2023-01-04 09:01:08 -05:00
Maria Khalusova
b493fee958 Add: doc page for the object detection task (#20925) 
* Added Object Detection task guide (new branch)

* Polished code examples after running make style

* Update docs/source/en/tasks/object_detection.mdx

Rephrasing suggestion from Sayak

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* Update docs/source/en/tasks/object_detection.mdx

A rephrasing suggestion from Sayak

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* Update docs/source/en/tasks/object_detection.mdx

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* Update docs/source/en/tasks/object_detection.mdx

typo

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* Update docs/source/en/tasks/object_detection.mdx

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* Update docs/source/en/tasks/object_detection.mdx

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* Update docs/source/en/tasks/object_detection.mdx

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* Update docs/source/en/tasks/object_detection.mdx

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update docs/source/en/tasks/object_detection.mdx

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update docs/source/en/tasks/object_detection.mdx

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update docs/source/en/tasks/object_detection.mdx

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update docs/source/en/tasks/object_detection.mdx

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update docs/source/en/tasks/object_detection.mdx

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update docs/source/en/tasks/object_detection.mdx

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update docs/source/en/tasks/object_detection.mdx

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update docs/source/en/tasks/object_detection.mdx

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Applied reviewers suggestions
>
>
Co-authored-by: sayakpaul <spsayakpaul@gmail.com>
Co-authored-by: sgugger <sylvain.gugger@gmail.com>

* polished code examples

* Added a visualization of the inference result. Slightly changed hyperparameters, and updated the results.

* polished code examples

* Update docs/source/en/tasks/object_detection.mdx

Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>

* Update docs/source/en/tasks/object_detection.mdx

Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>

* Applying Steven's review suggestions

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>

* minor punctuation fix

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
 2023-01-04 08:36:37 -05:00
Arthur
d7b66d9b44 update template (#20885) 
* update template

* replace redme entries

* make style
 2023-01-04 10:15:45 +01:00
Jongjyh
ce85686a1f Add AltCLIP (#20446) 
* add altclip

* update

* fix wrong title

* fix the copyright in readme

* add altclip model

* add altclip

* fix test_gradient_checkpointing_enable_disable

* code

* add return class

* add projection_state

* "fix pretrained model bug"

* delete print and fix 2 test instances.

* delete token

* rm xlmr

* one model one file.

* empty commit to trigger CI

* Fix modeling_outputs.py

* Fix __init__

* Fix quality

* Fix modeling file docstring

* Fix README.md

* Fix test file

* add vision model

* empty commit to trigger CI

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* del token in mdx file

* fix

* fix

* fix

* remove altrob from test list

* add vision test

* fix fx

* fix

* fix

* fix

* trigger CI

* fix copies

* fix tests

* fix style

* fix quality

* update

* recover import

* recover

* add ,

* recover

* fix copies

* trigger CI

* fix

* some of review

* update

* remove import

* last 2

* fix

* fix style

* fix style

* fix bug

* fix uncomment

* fix

* update

* fix

* second review

* empty commit to trigger CI

* empty commit to trigger CI

* fix position

* fix

* empty commit to trigger CI

* empty commit to trigger CI

* third comment

* Update docs/source/en/model_doc/altclip.mdx

Co-authored-by: Yih-Dar <2521628+ydshieh@users.noreply.github.com>

* Update docs/source/en/model_doc/altclip.mdx

Co-authored-by: Yih-Dar <2521628+ydshieh@users.noreply.github.com>

* Update src/transformers/__init__.py

Co-authored-by: Yih-Dar <2521628+ydshieh@users.noreply.github.com>

* Update src/transformers/models/altclip/configuration_altclip.py

Co-authored-by: Yih-Dar <2521628+ydshieh@users.noreply.github.com>

* Update src/transformers/models/altclip/modeling_altclip.py

Co-authored-by: Yih-Dar <2521628+ydshieh@users.noreply.github.com>

* Update src/transformers/models/altclip/processing_altclip.py

Co-authored-by: Yih-Dar <2521628+ydshieh@users.noreply.github.com>

* Update src/transformers/models/altclip/modeling_altclip.py

Co-authored-by: Yih-Dar <2521628+ydshieh@users.noreply.github.com>

* fix merge

* fix copies

* update

* update

* empty commit to trigger CI

* fix code example

* empty commit to trigger CI

* fix

* empty commit to trigger CI

* empty commit to trigger CI

Co-authored-by: shunxing1234 <xw747777271@gmail.com>
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
Co-authored-by: shunxing1234 <33774367+shunxing1234@users.noreply.github.com>
Co-authored-by: Yih-Dar <2521628+ydshieh@users.noreply.github.com>
 2023-01-04 09:18:57 +01:00
Motoki Wu
45da7cec5a Add custom stop token ids for generation (#20727) 
* Add StopIdStoppingCriteria

* add a working test for stop id criteria

* add to global scope

* add stop_ids to generate

* add pipeline test

* use tokenizer encode in test

* add test to generation utils

* reformat

* fixup

* make-fix-copies

* rename to stop_token_id

* use stop_tokens instead

* add to text to text generation

* make fixup

* make repo-consistency

* Add support for list of ints for eos_token_id inside generation/utils.py

* Instead of having if elses, cast the eos_token_id into a List[int]

* Add List[int] support for logits_process.py

* add List[int] for beam_search.py

* add List[int] for forced_eos_token_id

* revert stop token id stopping criteria changes

* make fixup

* fix tests

* add eos_token_id to generation/utils.py and added tests test_utils.py

* add eos_token_id type hints and fix for pad tokens

* add comments

* remove some prints and remove forced false test

* fix

* put back test_stop_sequence_stopping_criteria

* remove unused import and make fixup

* add a none check

* update docstring

* add more docstring for list ints

* make fixup
 2023-01-03 15:18:24 -05:00
radcheb
cd918492c6 Fix race condition on cleaning checkpoints when save_total_limit set to 1 (#20989) 
* Update trainer.py

* fix style

Co-authored-by: Radhwane Chebaane <rchebaane.external@epo.org>
 2023-01-03 15:16:12 -05:00
Alara Dirik
cd2457809f Improve OWL-ViT postprocessing (#20980) 
* add post_process_object_detection method

* style changes
 2023-01-03 19:25:09 +03:00
Yih-Dar
e901914da7 Fix for LXMERT (#20986) 
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-01-03 17:16:52 +01:00
Yih-Dar
8f09dd89f6 Avoid CI runs under users' own CircleCI personal account (#20981) 
* Avoid null CI

* Avoid null CI

* rename

* more clear error message

* Update .circleci/config.yml

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* clean up

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2023-01-03 16:19:38 +01:00
Anna Krogager
7b0727a401 Ignore errors when deleting old checkpoints in trainer (#20984) 2023-01-03 10:10:59 -05:00
samuelpullely
15c68c67f4 Enable decoder_attention_mask in generate function (#20726) 
* Enable `decoder_attention_mask` in `generate` function

* Make style corrections

* Run `make repo-consistency`

* Add integration test
 2023-01-03 09:59:08 -05:00
JeongYeon Nam
a9653400d3 Fix valid ratio for Deformable Detr (#20958) 
* fix: valid ratio has right value

* chore: remove unnecessary line

Co-authored-by: Jeongyeon Nam <jy.nam@navercorp.com>
 2023-01-03 09:43:26 -05:00
Wang, Yi
9c9fe89f84 [run_clm example] add torch_dtype option for model load. (#20971) 
* [run_clm example] add torch_dtype option for model load.
for BLOOM 175B model. peak memory will reduce about 350G for inference. the weight of BLOOM in model hub is bfloat16

Signed-off-by: Wang, Yi A <yi.a.wang@intel.com>

* add other type in option

* fix style

Signed-off-by: Wang, Yi A <yi.a.wang@intel.com>
 2023-01-03 09:33:11 -05:00
Yih-Dar
e697c912c2 Remove more unused attributes in config classes (#20858) 
Remove more unused attributes in config classes

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-01-03 14:37:40 +01:00
NielsRogge
9c6f7485a6 Add GIT (GenerativeImage2Text) (#20295) 
* First draft

* Make model instantiation work

* Fix copied from statement

* More fixes

* Add correct output head

* Improve configuration

* Add conversion script

* Improve conversion script

* Remove token_type_ids

* Fix conversion of projection layers

* Convert all weights

* Use cats image

* Make logits match

* Generate caption on cats image

* Add GITProcessor

* Update conversion script

* Add support for more checkpoints

* Fix conversion script

* Add initial tests

* Remove cross-attention

* More improvements

* Remove is_decoder

* Improve model tests

* Improve tests

* Improve model outputs

* Fix model outputs equivalence

* Fix more tests

* Remove unused code

* Use generate to generate text, no use of cache for now

* Use generate more appropriately

* Fix config tests

* Fix style

* Add support for use_cache

Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com>

* Fix style

* Fix GIT vision encoder

* Update README

* Fix integration test

* Set bos and eos token ids

* Improve docs

* Improve code

* Add support for provided attention_mask

* Add copied from statement

* Fix gradient checkpointing test

* Set model_input_names

* Investigate model_input_names

* Remove script

* Fix model inputs

* Fix docstring

* Rename GIT to Git

* Support more models

* Add support for textvqa model

* Add video support

* Extend conversion script for video

* Add support for large variant

* Add support for more models

* Fix config archive map

* Update integration test

* Fix README

* Fix CLIP mean and std

* Update processor

* Fix use_cache for video, thanks @gante

* Remove print statements

* Remove assertion

* Add processor tests

* Fix model_input_names

* Use Auto API for processor

* Fix processor tests

* Fix integration test

* Fix pipeline test

* Make tests faster

* Update conversion script

* Update conversion script

* Convert more checkpoints

* Update conversion script

* Fix typo

* Update docstrings

* Improve code snippets

* Fix doc tests

* Add more code examplesé

* Fix doc tests

* Add integration tests

* Fix unused variable

* revert

* Add GIT to Japanese README

Co-authored-by: Niels Rogge <nielsrogge@Nielss-MacBook-Pro.local>
Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com>
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-01-03 14:17:18 +01:00
Alara Dirik
305f41e4de Fix post_process_object_detection method descriptions (#20977) 
fix post_process_object_detection descriptions
 2023-01-03 15:56:02 +03:00