SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity
17,235 Commits 5 Branches 239 Tags
72c716de9274fce07924cb7780dc6392fc276923
Commit Graph

2527 Commits

Author SHA1 Message Date
dependabot[bot]
3a36597a5f Bump pillow from 10.0.1 to 10.2.0 in /examples/research_projects/decision_transformer (#28655) 
Bump pillow in /examples/research_projects/decision_transformer

Bumps [pillow](https://github.com/python-pillow/Pillow) from 10.0.1 to 10.2.0.
- [Release notes](https://github.com/python-pillow/Pillow/releases)
- [Changelog](https://github.com/python-pillow/Pillow/blob/main/CHANGES.rst)
- [Commits](https://github.com/python-pillow/Pillow/compare/10.0.1...10.2.0)

---
updated-dependencies:
- dependency-name: pillow
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-05-01 19:58:34 +01:00
dependabot[bot]
4f3c7af489 Bump torch from 1.9.0+cpu to 1.13.1 in /examples/research_projects/jax-projects/hybrid_clip (#21167) 
Bump torch in /examples/research_projects/jax-projects/hybrid_clip

Bumps [torch](https://github.com/pytorch/pytorch) from 1.9.0+cpu to 1.13.1.
- [Release notes](https://github.com/pytorch/pytorch/releases)
- [Changelog](https://github.com/pytorch/pytorch/blob/master/RELEASE.md)
- [Commits](https://github.com/pytorch/pytorch/commits/v1.13.1)

---
updated-dependencies:
- dependency-name: torch
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-05-01 18:37:55 +01:00
dependabot[bot]
6f465d45d9 Bump torch from 1.11.0 to 1.13.1 in /examples/research_projects/decision_transformer (#21171) 
Bump torch in /examples/research_projects/decision_transformer

Bumps [torch](https://github.com/pytorch/pytorch) from 1.11.0 to 1.13.1.
- [Release notes](https://github.com/pytorch/pytorch/releases)
- [Changelog](https://github.com/pytorch/pytorch/blob/master/RELEASE.md)
- [Commits](https://github.com/pytorch/pytorch/compare/v1.11.0...v1.13.1)

---
updated-dependencies:
- dependency-name: torch
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-05-01 18:16:25 +01:00
dependabot[bot]
6e0cba3cec Bump torch from 1.6.0 to 1.13.1 in /examples/research_projects/visual_bert (#21172) 
Bump torch in /examples/research_projects/visual_bert

Bumps [torch](https://github.com/pytorch/pytorch) from 1.6.0 to 1.13.1.
- [Release notes](https://github.com/pytorch/pytorch/releases)
- [Changelog](https://github.com/pytorch/pytorch/blob/master/RELEASE.md)
- [Commits](https://github.com/pytorch/pytorch/compare/v1.6.0...v1.13.1)

---
updated-dependencies:
- dependency-name: torch
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-05-01 16:40:54 +01:00
dependabot[bot]
ce66c0e989 Bump torch from 1.11.0 to 1.13.1 in /examples/research_projects/codeparrot (#21170) 
Bump torch in /examples/research_projects/codeparrot

Bumps [torch](https://github.com/pytorch/pytorch) from 1.11.0 to 1.13.1.
- [Release notes](https://github.com/pytorch/pytorch/releases)
- [Changelog](https://github.com/pytorch/pytorch/blob/master/RELEASE.md)
- [Commits](https://github.com/pytorch/pytorch/compare/v1.11.0...v1.13.1)

---
updated-dependencies:
- dependency-name: torch
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-05-01 16:40:19 +01:00
dependabot[bot]
7a29c577e8 Bump torch from 1.6.0 to 1.13.1 in /examples/research_projects/lxmert (#21174) 
Bumps [torch](https://github.com/pytorch/pytorch) from 1.6.0 to 1.13.1.
- [Release notes](https://github.com/pytorch/pytorch/releases)
- [Changelog](https://github.com/pytorch/pytorch/blob/master/RELEASE.md)
- [Commits](https://github.com/pytorch/pytorch/compare/v1.6.0...v1.13.1)

---
updated-dependencies:
- dependency-name: torch
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-05-01 16:39:55 +01:00
dependabot[bot]
b33f01fe6b Bump pyarrow from 1.0.1 to 15.0.0 in /examples/research_projects/lxmert (#30584) 
Bumps [pyarrow](https://github.com/apache/arrow) from 1.0.1 to 15.0.0.
- [Commits](https://github.com/apache/arrow/compare/apache-arrow-1.0.1...go/v15.0.0)

---
updated-dependencies:
- dependency-name: pyarrow
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-05-01 16:38:07 +01:00
dependabot[bot]
0ec3003ae9 Bump pyarrow from 1.0.1 to 15.0.0 in /examples/research_projects/visual_bert (#30583) 
Bump pyarrow in /examples/research_projects/visual_bert

Bumps [pyarrow](https://github.com/apache/arrow) from 1.0.1 to 15.0.0.
- [Commits](https://github.com/apache/arrow/compare/apache-arrow-1.0.1...go/v15.0.0)

---
updated-dependencies:
- dependency-name: pyarrow
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-05-01 16:37:54 +01:00
dependabot[bot]
aefbdfe8cf Bump pyarrow from 7.0.0 to 15.0.0 in /examples/research_projects/decision_transformer (#30582) 
Bump pyarrow in /examples/research_projects/decision_transformer

Bumps [pyarrow](https://github.com/apache/arrow) from 7.0.0 to 15.0.0.
- [Commits](https://github.com/apache/arrow/compare/go/v7.0.0...go/v15.0.0)

---
updated-dependencies:
- dependency-name: pyarrow
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-05-01 16:37:40 +01:00
dependabot[bot]
7164171212 Bump gitpython from 3.1.32 to 3.1.41 in /examples/research_projects/distillation (#30586) 
Bump gitpython in /examples/research_projects/distillation

Bumps [gitpython](https://github.com/gitpython-developers/GitPython) from 3.1.32 to 3.1.41.
- [Release notes](https://github.com/gitpython-developers/GitPython/releases)
- [Changelog](https://github.com/gitpython-developers/GitPython/blob/main/CHANGES)
- [Commits](https://github.com/gitpython-developers/GitPython/compare/3.1.32...3.1.41)

---
updated-dependencies:
- dependency-name: gitpython
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-05-01 16:36:57 +01:00
dependabot[bot]
ff8f624542 Bump grpcio from 1.44.0 to 1.53.2 in /examples/research_projects/decision_transformer (#30585) 
Bump grpcio in /examples/research_projects/decision_transformer

Bumps [grpcio](https://github.com/grpc/grpc) from 1.44.0 to 1.53.2.
- [Release notes](https://github.com/grpc/grpc/releases)
- [Changelog](https://github.com/grpc/grpc/blob/master/doc/grpc_release_schedule.md)
- [Commits](https://github.com/grpc/grpc/compare/v1.44.0...v1.53.2)

---
updated-dependencies:
- dependency-name: grpcio
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-05-01 16:35:52 +01:00
dependabot[bot]
b71f512823 Bump gitpython from 3.1.32 to 3.1.41 in /examples/research_projects/decision_transformer (#30587) 
Bump gitpython in /examples/research_projects/decision_transformer

Bumps [gitpython](https://github.com/gitpython-developers/GitPython) from 3.1.32 to 3.1.41.
- [Release notes](https://github.com/gitpython-developers/GitPython/releases)
- [Changelog](https://github.com/gitpython-developers/GitPython/blob/main/CHANGES)
- [Commits](https://github.com/gitpython-developers/GitPython/compare/3.1.32...3.1.41)

---
updated-dependencies:
- dependency-name: gitpython
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-05-01 16:30:24 +01:00
amyeroberts
bbaa8ceff6 Fix canonical model --model_type in examples (#30480) 
Fix --model_type in examples
 2024-05-01 15:47:05 +01:00
Matt
1e05671d21 Fix QA example (#30580) 
* Handle cases when CLS token is absent

* Use BOS token as a fallback
 2024-05-01 08:43:02 +01:00
Anton Vlasjuk
9112520b15 Fix seq2seq collator padding (#30556) 
* fix seq2seq data collator to respect the given padding strategy

further added tests for the seq2seq data collator in the style of the `data_collator_for_token_classification` (pt, tf, np)

* formatting and change bool equals "==" to "is"

* add missed return types in tests

* update numpy test as it can handle unequal shapes, not like pt or tf
 2024-04-30 18:32:30 +01:00
Sanchit Gandhi
38b53da38a [examples] update whisper fine-tuning (#29938) 
* [examples] update whisper fine-tuning

* deprecate forced/suppress tokens

* item assignment

* update readme

* final fix
 2024-04-26 17:06:03 +01:00
Arthur
661190b44d [research_project] Most of the security issues come from this requirement.txt (#29977) 
update most of decision transformers research project
 2024-04-24 17:56:45 +02:00
Zach Mueller
60d5f8f9f0 🚨🚨🚨Deprecate evaluation_strategy to eval_strategy🚨🚨🚨 (#30190) 
* Alias

* Note alias

* Tests and src

* Rest

* Clean

* Change typing?

* Fix tests

* Deprecation versions
 2024-04-18 12:49:43 -04:00
Lysandre
ce8e64fbe2 Dev version 2024-04-18 15:53:25 +02:00
Nicolas Patry
8e5f76f511 Upgrading to tokenizers 0.19.0 (#30289) 
* [DO NOT MERGE] Testing tokenizers 0.19.0rc0

* Accounting for the breaking change.

* Ruff.

* Upgrading to tokenizers `0.19` (new release with preprend_scheme fixed
and new surface for BPE tiktoken bug).
 2024-04-17 17:17:50 +02:00
Yih-Dar
05dab4e5ba Fix test ExamplesTests::test_run_translation (#30281) 
fix

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2024-04-17 11:46:33 +02:00
Sai-Suraj-27
b3595cf02b fix: Replaced deprecated typing.Text with str (#30230) 
typing.Text is deprecated. Use str instead
 2024-04-15 12:18:37 +01:00
JINO ROHIT
f010786218 Set pad_token in run_glue_no_trainer.py #28534 (#30234) 2024-04-15 11:39:10 +01:00
Pavel Iakubovskii
56d001b26f Fix and simplify semantic-segmentation example (#30145) 
* Remove unused augmentation

* Fix pad_if_smaller() and remove unused augmentation

* Add indentation

* Fix requirements

* Update dataset use instructions

* Replace transforms with albumentations

* Replace identity transform with None

* Fixing formatting

* Fixed comment place
 2024-04-10 09:10:52 +01:00
NielsRogge
e9c23fa056 [Trainer] Undo #29896 (#30129) 
* Undo

* Use tokenizer

* Undo data collator
 2024-04-09 12:55:42 +02:00
JINO ROHIT
f5658732d5 fixing issue 30034 - adding data format for run_ner.py (#30088) 2024-04-08 12:49:59 +01:00
Haz Sameen Shahgir
5e673ed2dc updated examples/pytorch/language-modeling scripts and requirements.txt to require datasets>=2.14.0 (#30120) 
updated requirements.txt and require_version() calls in examples/pytorch/language-modeling to require datasets>=2.14.0
 2024-04-08 12:41:28 +01:00
NielsRogge
1ab7136488 [Trainer] Allow passing image processor (#29896) 
* Add image processor to trainer

* Replace tokenizer=image_processor everywhere
 2024-04-05 10:10:44 +02:00
Mario Šaško
fce52cefa7 Fix remove_columns in text-classification example (#29351) 2024-04-02 19:15:27 +02:00
Jacky Lee
156d30da94 Add warning message for run_qa.py (#29867) 
* improve: error message for best model metric

* update: raise warning instead of error
 2024-03-30 17:02:31 +01:00
Lysandre Debut
39114c0383 Remove static pretrained maps from the library's internals (#29112) 
* [test_all] Remove static pretrained maps from the library's internals

* Deprecate archive maps instead of removing them

* Revert init changes

* [test_all] Deprecate instead of removing

* [test_all] PVT v2 support

* [test_all] Tests should all pass

* [test_all] Style

* Address review comments

* Update src/transformers/models/deprecated/_archive_maps.py

Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>

* Update src/transformers/models/deprecated/_archive_maps.py

Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>

* [test_all] trigger tests

* [test_all] LLAVA

* [test_all] Bad rebase

---------

Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
 2024-03-25 10:33:38 +01:00
Jacky Lee
ef6e371dba Add support for torch_dtype in the run_mlm example (#29776) 
feat: add support for torch_dtype

Co-authored-by: Jacky Lee <jackylee328@gmail.com>
 2024-03-21 15:09:35 +00:00
Arthur Zucker
1248f09252 v4.40.0.dev.0 2024-03-20 23:31:47 +09:00
Quentin Lhoest
f02aea2737 Rename glue to nyu-mll/glue (#29679) 
* Update run_glue.py

* Update run_glue.py

* Update run_glue_no_trainer.py
 2024-03-15 16:35:02 +01:00
guangwu
03847ef451 fix: typos (#29653) 
Signed-off-by: guoguangwu <guoguangwug@gmail.com>
 2024-03-15 15:02:50 +00:00
Joao Gante
d47966536c Examples: check max_position_embeddings in the translation example (#29600) 
check max_position_embeddings
 2024-03-12 18:58:12 +00:00
Hilco van der Wilk
b6404866cd Update legacy Repository usage in various example files (#29085) 
* Update legacy Repository usage in `examples/pytorch/text-classification/run_glue_no_trainer.py`

Marked for deprecation here https://huggingface.co/docs/huggingface_hub/guides/upload#legacy-upload-files-with-git-lfs

* Fix import order

* Replace all example usage of deprecated Repository

* Fix remaining repo call and rename args variable

* Revert removing creation of gitignore files and don't change research examples
 2024-03-12 13:20:49 +00:00
Matt
81ec8028f9 Stop passing None to compile() in TF examples (#29597) 
* Fix examples to stop passing None to compile(), rework example invocation for run_text_classification.py

* Add Amy's fix
 2024-03-12 12:22:29 +00:00
Yitong Huang
873d9bb3cc Make torch xla available on GPU (#29334) 
* add USE_TORCH_XLA env

* rename torch_tpu to torch_xla

* better is_torch_xla_available; fix some fsdp and performance issues

* fix format

* fix bug when pjrt_device is cpu

* fix bug

* fix the deprecation handling

---------

Co-authored-by: anw90 <ang868@gmail.com>
Co-authored-by: wangang.wa <wangang.wa@alibaba-inc.com>
 2024-03-11 14:07:16 +00:00
Tanay Mehta
6d67837f06 Add Fill-in-the-middle training objective example - PyTorch (#27464) 
* add: initial script to train clm fim

* fix: if training model from scratch, new tokens will be added and embeddings resized

* fix: fixed attention_mask errors when generating FIM data

* fix: file formatted using black

* add: run_fim_no_trainer.py and fixed some comments in run_fim.py

* add: added fim examples to the README.md and ran code fixup

* fix: little bug in both fim training scripts

* fix: remove comment from notebook and added a note on fim related params

* fix: minor typo in README

* add: suggested minor changes to README and run_fim.py

* add: gradient_accumulation_steps and gradient_checkpointing args

* add: improved model embedding resizing

* add: pad_to_multiple_of and attn_implementation params

* add: requested minor changes

* add: deepspeed zero compatibility

* add: resize embeddings layer with zero3 support for fim model initialization
 2024-03-11 12:14:02 +00:00
Arthur Zucker
1a77f07f65 v4.39.dev.0 2024-02-21 15:23:22 +09:00
Arthur
b2724d7b4c change version (#29097) 
* change version

* nuke

* this doesn't make sense

* update some requirements.py

* revert + no main

* nits

* change cache number

* more pin

* revert

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2024-02-19 22:08:44 +08:00
Jay Zhou
79132d4cfe Fix a typo in examples/pytorch/text-classification/run_classification.py (#29072) 2024-02-19 13:01:15 +00:00
Lysandre Debut
f497f564bb Update all references to canonical models (#29001) 
* Script & Manual edition

* Update
 2024-02-16 08:16:58 +01:00
Klaus Hipp
fe3df9d5b3 [Docs] Add language identifiers to fenced code blocks (#28955) 
Add language identifiers to code blocks
 2024-02-12 10:48:31 -08:00
Alexey Fadeev
792819f6cf Updated requirements for image-classification samples: datasets>=2.14.0 (#28974) 
Updated datasets requirements. Need a package version >= 2.14.0
 2024-02-12 14:57:25 +00:00
Yih-Dar
308d2b9004 Update the cache number (#28905) 
* fix

* fix

* fix

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2024-02-07 16:37:09 +01:00
Klaus Hipp
1c31b7aa3b [Docs] Add missing language options and fix broken links (#28852) 
* Add missing entries to the language selector

* Add links to the Colab and AWS Studio notebooks for ONNX

* Use anchor links in CONTRIBUTING.md

* Fix broken hyperlinks due to spaces

* Fix links to OpenAI research articles

* Remove confusing footnote symbols from author names, as they are also considered invalid markup
 2024-02-06 12:01:01 -08:00
dependabot[bot]
e83227d76e Bump cryptography from 41.0.2 to 42.0.0 in /examples/research_projects/decision_transformer (#28879) 
Bump cryptography in /examples/research_projects/decision_transformer

Bumps [cryptography](https://github.com/pyca/cryptography) from 41.0.2 to 42.0.0.
- [Changelog](https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pyca/cryptography/compare/41.0.2...42.0.0)

---
updated-dependencies:
- dependency-name: cryptography
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-02-06 03:53:08 +01:00
dependabot[bot]
ca8944c4e3 Bump dash from 2.3.0 to 2.15.0 in /examples/research_projects/decision_transformer (#28845) 
Bump dash in /examples/research_projects/decision_transformer

Bumps [dash](https://github.com/plotly/dash) from 2.3.0 to 2.15.0.
- [Release notes](https://github.com/plotly/dash/releases)
- [Changelog](https://github.com/plotly/dash/blob/dev/CHANGELOG.md)
- [Commits](https://github.com/plotly/dash/compare/v2.3.0...v2.15.0)

---
updated-dependencies:
- dependency-name: dash
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-02-05 03:12:30 +01:00