SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity
16,219 Commits 5 Branches 239 Tags
1218e439b5fe05423693407653a1fb064263fef4
Commit Graph

16219 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Yih-Dar
1f84141391 Fix nightly circleci (#31114) 
* fix

* fix

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2024-05-29 15:42:39 +02:00
Zach Mueller
d16053c867 Rm maintainer + migrate (#31089) 2024-05-29 09:35:37 -04:00
Matt
0bef4a2738 Fix faulty rstrip in module loading (#31108) 2024-05-29 13:33:26 +01:00
Matt
97a58a5d2c Fix env.py in cases where torch is not present (#31113) 
* Fix env.py in cases where torch is not present

* Simplify the fix (and avoid some issues)
 2024-05-29 13:20:36 +01:00
Huazhong Ji
c8861376ad Improve transformers-cli env reporting (#31003) 
* Improve `transformers-cli env` reporting

* move the line `"Using GPU in script?": "<fill in>"` to in if conditional
statement

* same option for npu
 2024-05-29 11:57:54 +01:00
Lucain
c3044ec2f3 Use HF_HUB_OFFLINE + fix has_file in offline mode (#31016) 
* Fix has_file in offline mode

* harmonize env variable for offline mode

* Switch to HF_HUB_OFFLINE

* fix test

* revert test_offline to test TRANSFORMERS_OFFLINE

* Add new offline test

* merge conflicts

* docs
 2024-05-29 11:55:43 +01:00
Younes Belkada
bfe6f513b9 FEAT: Add mistral v3 conversion script (#30981) 
* add mistral v3 conversion script

* Update src/transformers/models/mistral/convert_mistral_weights_to_hf.py

Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>

* fixup

---------

Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
 2024-05-29 11:43:54 +02:00
Raushan Turganbay
d521ba5797 Quantized KV cache: update quanto (#31052) 
* quanto latest version was refactored

* add error msg

* incorrect compare sign

* Update src/transformers/cache_utils.py

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

---------

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
 2024-05-29 14:25:44 +05:00
amyeroberts
a564d10afe Deprecate low use models (#30781) 
* Deprecate models
- graphormer
- time_series_transformer
- xlm_prophetnet
- qdqbert
- nat
- ernie_m
- tvlt
- nezha
- mega
- jukebox
- vit_hybrid
- x_clip
- deta
- speech_to_text_2
- efficientformer
- realm
- gptsan_japanese

* Fix up

* Fix speech2text2 imports

* Make sure message isn't indented

* Fix docstrings

* Correctly map for deprecated models from model_type

* Uncomment out

* Add back time series transformer and x-clip

* Import fix and fix-up

* Fix up with updated ruff
 2024-05-28 18:07:07 +01:00
Younes Belkada
7f08817be4 Docs / Quantization: Redirect deleted page (#31063) 
Update _redirects.yml
 2024-05-28 18:29:22 +02:00
Younes Belkada
3264be4114 TST: Fix instruct-blip tests (#31088) 
* fix flan t5 tests

* better format
 2024-05-28 18:29:11 +02:00
Jonny Li
476890e9ae Fix DeepSpeed compatibility with weight_norm (#30881) (#31018) 2024-05-28 17:25:15 +01:00
Albert Villanova del Moral
aada568f73 Fix PretrainedConfig docstring with deprecated resume_download (#31014) 2024-05-28 17:47:35 +02:00
Yih-Dar
3af7bf30ad skip test_multi_gpu_data_parallel_forward for vit and deit (#31086) 
fix

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2024-05-28 17:44:52 +02:00
Younes Belkada
ab19f907fd FIX / OPT: Fix OPT multi-GPU training for OPTForQuestionAnswering (#31092) 
Update modeling_opt.py
 2024-05-28 17:06:00 +02:00
Younes Belkada
94d416f018 FIX: Add accelerate as a hard requirement (#31090) 
add accelerate
 2024-05-28 17:05:44 +02:00
Sigbjørn Skjæret
22dab246c5 Render chat template tojson filter as unicode (#31041) 
* Render chat template tojson filter as unicode

* ruff--
 2024-05-28 15:02:51 +01:00
Younes Belkada
4f98b14465 Docs / PEFT: Add PEFT API documentation (#31078) 
* add peft references

* add peft references

* Update docs/source/en/peft.md

* Update docs/source/en/peft.md
 2024-05-28 15:04:43 +02:00
Raushan Turganbay
779bc360ff Watermark: fix tests (#30961) 
* fix tests

* style

* Update tests/generation/test_utils.py

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

---------

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
 2024-05-28 17:07:42 +05:00
Lysandre Debut
a3c7b59e31 Fix failing tokenizer tests (#31083) 
* Fix failing tokenizer tests

* Use small tokenizer

* Fix remaining reference
 2024-05-28 13:34:23 +02:00
NielsRogge
90da0b1c9f [SuperPoint, PaliGemma] Update docs (#31025) 
* Update docs

* Add PaliGemma resources

* Address comment

* Update docs
 2024-05-28 13:22:06 +02:00
Sina Taslimi
66add161dc Fix typo in trainer.py (#31048) 2024-05-28 12:09:32 +01:00
Pavel Iakubovskii
98e2d48e9a Fix OWLv2 post_process_object_detection for multiple images (#31082) 
* Add test for multiple images

* [run slow] owlv2

* Fix box rescaling

* [run slow] owlv2
 2024-05-28 12:06:06 +01:00
Pavel Iakubovskii
c31473ed44 Remove float64 cast for OwlVit and OwlV2 to support MPS device (#31071) 
Remove float64
 2024-05-28 11:41:40 +01:00
oOraph
936ab7bae5 fix from_pretrained in offline mode when model is preloaded in cache (#31010) 
* Unit test to verify fix

Signed-off-by: Raphael Glon <oOraph@users.noreply.github.com>

* fix from_pretrained in offline mode when model is preloaded in cache

Signed-off-by: Raphael Glon <oOraph@users.noreply.github.com>

* minor: fmt

Signed-off-by: Raphael Glon <oOraph@users.noreply.github.com>

---------

Signed-off-by: Raphael Glon <oOraph@users.noreply.github.com>
Co-authored-by: Raphael Glon <oOraph@users.noreply.github.com>
 2024-05-28 11:56:05 +02:00
Hengwen Tong
537deb7869 Remove redundant backend checks in training_args.py (#30999) 
* Remove backend checks in training_args.py

* Expilicit initialize the device

---------

Co-authored-by: tonghengwen <tonghengwen@cambricon.com>
 2024-05-28 11:52:47 +02:00
AP
dd4654eab7 Update quicktour.md to fix broken link to Glossary (#31072) 
Update quicktour.md to fix broken link

Missing '/' in attention mask link in the transformers quicktour
 2024-05-28 11:50:45 +02:00
Clint Adams
e18da4e3f2 fix "piano" typo (#31027) 2024-05-28 11:48:23 +02:00
Yih-Dar
8e3b1fef97 Remove ninja from docker image build (#31080) 
fix

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2024-05-28 11:36:26 +02:00
Yih-Dar
8f0f7271d0 use @main (#31065) 
use main

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2024-05-28 10:53:28 +02:00
Yih-Dar
9d35edbb30 skip test_model_parallelism for 2 model test classes (#31067) 
skip

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2024-05-27 18:36:39 +02:00
Yoach Lacombe
d355741eca Fix pad_to_max_length Whisper (#30787) 
* fix pad_to_max_length Whisper

* add tests

* make style
 2024-05-27 16:09:05 +02:00
Marc Sun
b84cd67526 Fix quanto tests (#31062) 
fix quanto tests
 2024-05-27 15:53:45 +02:00
amyeroberts
cd797778e4 Update feature request label in template (#30940) 2024-05-27 15:16:47 +02:00
Eitan Turok
0a064dc0fc Follow up: Fix link in dbrx.md (#30514) 
* Fix link in dbrx.md

* remove "though this may not be up to date"

---------

Co-authored-by: Lysandre Debut <hi@lysand.re>
 2024-05-27 14:57:43 +02:00
Yih-Dar
d7942d9d27 unpin uv (#31055) 
[push-ci-image]

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2024-05-27 13:47:47 +02:00
Aymeric Roucher
84c4b72ee9 Redirect transformers_agents doc to agents (#31054) 2024-05-27 10:34:14 +02:00
Pablo Montalvo
bdb9106f24 Paligemma- fix devices and dtype assignments (#31008) 
* fix devices and dtype assignments

* [run-slow]paligemma
 2024-05-24 19:02:55 +02:00
Ita Zaporozhets
deba7655e6 Add split special tokens (#30772) 
* seems like `split_special_tokens` is used here

* split special token

* add new line at end of file

* moving split special token test to common tests

* added assertions

* test

* fixup

* add co-author

* passing rest of args to gptsan_japanese, fixing tests

* removing direct comparison of fast and slow models

* adding test support for UDOP and LayoutXLM

* ruff fix

* readd check if slow tokenizer

* modify test to handle bos tokens

* removing commented function

* trigger build

* applying review feedback - updated docstrings, var names, and simplified tests

* ruff fixes

* Update tests/test_tokenization_common.py

Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>

* applying feedback, comments

* shutil temp directory fix

---------

Co-authored-by: Arthur Zucker <arthur.zucker@gmail.com>
Co-authored-by: Ita Zaporozhets <itazaporozhets@Itas-MBP.localdomain>
Co-authored-by: itazap <itazap@users.noreply.github.com>
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
Co-authored-by: Ita Zaporozhets <itazaporozhets@Itas-MacBook-Pro.local>
 2024-05-24 08:38:58 -07:00
BHUVAN M
e5103a76cc added interpolation for vitmae model in pytorch as well as tf. (#30732) 
* added interpolation for vitmae model in pytorch as well as tf.

* Update modeling_vit_mae.py

irreugalr import fixed

* small changes and proper formatting

* changes suggested in review.

* modified decoder interpolate_func

* arguments and docstring fix

* Apply suggestions from code review

doc fixes

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

---------

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
 2024-05-24 16:20:09 +01:00
Yih-Dar
a3cdff417b save the list of new model failures (#31013) 
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2024-05-24 15:20:25 +02:00
Younes Belkada
658b849aeb Quantization / TST: Fix remaining quantization tests (#31000) 
* Fix remaining quant tests

* Update test_quanto.py
 2024-05-24 14:35:59 +02:00
Lucain
fd3c128040 Fix resume_download future warning (#31007) 
* Fix resume_download future warning

* better like this

* Add regression test
 2024-05-24 14:35:40 +02:00
Yih-Dar
acbfaf69cc allow multi-gpu (#31011) 
* allow multi-gpu

* allow multi-gpu

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2024-05-24 14:20:06 +02:00
Marc Sun
ae87f9797b FIX / TST: Fix expected results on Mistral AWQ test (#30971) 
fix awq mistral test
 2024-05-24 14:06:31 +02:00
Fanli Lin
04c7c176d7 [tests] make test_model_parallelism device-agnostic (#30844) 
* enable on xpu

* fix style

* add comment and mps
 2024-05-24 11:51:51 +01:00
Yixiang Gao
42d8dd8716 Perceiver interpolate position embedding (#30979) 
* add test that currently fails

* test passed

* all perceiver passed

* fixup, style, quality, repo-consistency, all passed

* Apply suggestions from code review: default to False + compute sqrt once only

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

* fix a minor bracket

* replace dim with self._num_channels

* add arguments to the rest preprocessors

---------

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
 2024-05-24 11:13:58 +01:00
Yih-Dar
5855afd1f3 pin uv==0.1.45 (#31006) 
* fix

* [push-ci-image]

* run with latest

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2024-05-24 12:00:50 +02:00
Lucain
03935d300d Do not trigger autoconversion if local_files_only (#31004) 2024-05-24 11:00:59 +02:00
Kevin Koehncke
21e259d8c5 Fix training speed regression introduced by "optimize VRAM for calculating pos_bias in LayoutLM v2, v3 (#26139)" (#30988) 
* Revert "optimize VRAM for calculating pos_bias in LayoutLM v2, v3 (#26139)"

This reverts commit a7e0ed829c.

* Instead of reverting commit, wrap indexing in torch.no_grad context

* Apply wrapping in LayoutLMv2

* Add comments explaining reason for no_grad

* Fix code format

---------

Co-authored-by: Kevin Koehncke <kevin.koehncke@uipath.com>
 2024-05-24 10:43:44 +02:00