Yih-Dar
bbc00046b9
Fix flaky test_custom_4d_attention_mask ( #35606 )
...
* fix
* fix
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com >
2025-01-10 15:40:04 +01:00
Arthur Zucker
f63829c87b
v4.49.0-dev
2025-01-10 12:31:11 +01:00
Raushan Turganbay
52e1f87c7d
[WIP] Emu3: add model ( #33770 )
...
* model can convert to HF and be loaded back
* nit
* works in single batch generation but hallucinates
* use the image tokens
* add image generation
* now it works
* add tests
* update
* add modulare but it doesn't work for porting docstring :(
* skip some tests
* add slow tests
* modular removed the import?
* guess this works
* update
* update
* fix copies
* fix test
* fix copies
* update
* docs
* fix tests
* last fix tests?
* pls
* repo consistency
* more style
* style
* remove file
* address comments
* tiny bits
* update after the new modular
* fix tests
* add one more cond in check attributes
* decompose down/up/mid blocks
* allow static cache generation in VLMs
* nit
* fix copies
* Update docs/source/en/model_doc/emu3.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update docs/source/en/model_doc/emu3.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update docs/source/en/model_doc/emu3.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update docs/source/en/model_doc/emu3.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update docs/source/en/model_doc/emu3.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update docs/source/en/model_doc/emu3.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update docs/source/en/model_doc/emu3.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update docs/source/en/model_doc/emu3.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* fix VAE upsampling
* Update src/transformers/models/emu3/modular_emu3.py
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com >
* address comments
* state overwritten stuff explicitly
* fix copies
* add the flag for flex attn
---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com >
2025-01-10 12:23:00 +01:00
Cyril Vallez
ccc0381d36
Fix flex_attention in training mode ( #35605 )
...
* fix flex
* add test
* style
2025-01-10 11:49:12 +01:00
Arthur Zucker
a9bd1e6284
Remove benchmark.py after #34275
2025-01-10 11:09:06 +01:00
Raushan Turganbay
e0646f3dce
Chat template: return vectorized output in processors ( #34275 )
...
* update chat template
* style
* fix tests
* Update src/transformers/image_utils.py
Co-authored-by: Pavel Iakubovskii <qubvel@gmail.com >
* typehints + docs
* fix tests
* remove unnecessary warnings
* forgot code style :(
* allow users to pass backend and num frames
* Update docs/source/en/chat_templating.md
Co-authored-by: Pavel Iakubovskii <qubvel@gmail.com >
* Update src/transformers/image_utils.py
Co-authored-by: Pavel Iakubovskii <qubvel@gmail.com >
* Update src/transformers/image_utils.py
Co-authored-by: Pavel Iakubovskii <qubvel@gmail.com >
* Update src/transformers/image_utils.py
Co-authored-by: Pavel Iakubovskii <qubvel@gmail.com >
* Update src/transformers/image_utils.py
Co-authored-by: Pavel Iakubovskii <qubvel@gmail.com >
* Update src/transformers/image_utils.py
Co-authored-by: Pavel Iakubovskii <qubvel@gmail.com >
* Update src/transformers/image_utils.py
Co-authored-by: Pavel Iakubovskii <qubvel@gmail.com >
* Update src/transformers/processing_utils.py
Co-authored-by: Pavel Iakubovskii <qubvel@gmail.com >
* typo fix
* style
* address comments
* align with "pipeline" template
* update docs
* update docs
* unpack for all kwargs?
* wrong conflict resolution while rebasing
* tmp
* update docs
* Update docs/source/en/chat_templating.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update docs/source/en/chat_templating.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update docs/source/en/chat_templating.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update docs/source/en/chat_templating.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
---------
Co-authored-by: Pavel Iakubovskii <qubvel@gmail.com >
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
2025-01-10 11:05:29 +01:00
eustlb
5f087d1335
Add Moonshine ( #34784 )
...
* config draft
* full encoder forward
* full decoder forward
* fix sdpa and FA2
* fix sdpa and FA2
* moonshine model
* moonshine model forward
* fix attention with past_key_values
* add MoonshineForConditionalGeneration
* fix cache handling and causality for cross attention
* no causal attention mask for the encoder
* model addition (imports etc)
* small nit
* nits
* Update src/transformers/models/moonshine/convert_usefulsensors_to_hf.py
Co-authored-by: Joshua Lochner <admin@xenova.com >
* add rope_theta
* nits
* model doc
* Update src/transformers/models/auto/configuration_auto.py
Co-authored-by: Joshua Lochner <admin@xenova.com >
* imports
* add MODEL_FOR_SPEECH_SEQ_2_SEQ_MAPPING_NAMES
* updates modular
* make
* make fix-copies
* ruff check examples fix
* fix check_modular_conversion
* nit
* nits
* nits
* copied from -> imports
* imports fix
* integrate attention refacto
* modular edge case
* remove encoder
* convolutions params in config
* run modular_model_converter
* make
* Update docs/source/en/model_doc/moonshine.md
Co-authored-by: Joshua Lochner <admin@xenova.com >
* MoonshineModelTest
* correct typo
* make style
* integration tests
* make
* modular convert
* name conversion update (up_proj -> fc1 etc)
* update config
* update MLP
* update attention
* update encoder layer
* update decoder layer
* update convolutions parameters
* update encoder
* remove INPUTS_DOCSTRING
* update decoder
* update conditional generation
* update pretrained model
* imports
* modular converted
* update doc
* fix
* typo
* update doc
* update license
* update init
* split config in file
* two classes for MLP
* attention from GLM
* from GlmRotaryEmbedding
* split MLP
* apply arthur's review suggestions
* apply arthur's review suggestions
* apply arthur's review suggestions
* auto feature extractor
* convert modular
* fix + make
* convert modular
* make
* unsplit config
* use correct checkpoint
* wrap generate
* update tests
* typos
* make
* typo
* update doc
---------
Co-authored-by: Joshua Lochner <admin@xenova.com >
2025-01-10 11:00:54 +01:00
Yih-Dar
6f127d3f81
Skip torchscript tests if a cache object is in model's outputs ( #35596 )
...
* fix 1
* fix 1
* comment
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com >
2025-01-10 10:46:03 +01:00
Tom Aarsen
6b73ee8905
ModernBert: reuse GemmaRotaryEmbedding via modular + Integration tests ( #35459 )
...
* Introduce 5 integration tests for the 4 model classes + torch export
* ModernBert: reuse GemmaRotaryEmbedding via modular
* Revert #35589 , keep rope_kwargs; rely on them in modular_modernbert
* Revert "Revert #35589 , keep rope_kwargs; rely on them in modular_modernbert"
This reverts commit 11b44b9ee83e199cbfb7c5ba2d11f7a7fdbba2d3.
* Don't set rope_kwargs; override 'self.rope_init_fn' call instead
2025-01-10 10:25:10 +01:00
Zach Mueller
8de7b1ba8d
Add flex_attn to diffllama ( #35601 )
...
Add sdpa to diffllama
2025-01-09 20:49:11 +01:00
Benjamin Warner
1e3ddcb2d0
ModernBERT bug fixes ( #35404 )
...
* bug fixes
* organize imports
* wrap cpu warning in reference_compile
* Avoid needing repad_logits_with_grad, always repad with grads when training
I'm not 100% that the conditional with "or labels is None" makes sense though - not sure what the intention is there. Perhaps we can remove that?
* Revert "Avoid needing repad_logits_with_grad, always repad with grads when training"
This reverts commit cedcb4e89bcea199a1135a0933e71f534b656239.
* Fix grammar: keep -> keeps
* Propagate grammar fix with modular_model_converter
---------
Co-authored-by: Tom Aarsen <Cubiegamedev@gmail.com >
Co-authored-by: Tom Aarsen <37621491+tomaarsen@users.noreply.github.com >
2025-01-09 20:15:38 +01:00
Arthur
e97d7a5be5
add _supports_flex_attn = True for models that do support it ( #35598 )
...
* add `_supports_flex_attn = True`
* fix repo consistency
2025-01-09 20:03:33 +01:00
胡译文
c9c682d19c
[doc] deepspeed universal checkpoint ( #35015 )
...
* universal checkpoint
* Update docs/source/en/deepspeed.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update docs/source/en/deepspeed.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update docs/source/en/deepspeed.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
2025-01-09 09:50:51 -08:00
Cyril Vallez
3a4ae6eace
Refactor/fix Cohere2 ( #35594 )
...
* refactor/fix cohere2
* add kwargs
* tests
* remove func and import it
2025-01-09 17:54:57 +01:00
Tom Aarsen
32e0db8a69
[tokenizers] Ensure that add_prefix_space is propagated to backend_tokenizer.pre_tokenizer ( #35593 )
...
* Ensure that add_prefix_space is propagated to backend_tokenizer.pre_tokenizer
in PreTrainedTokenizerFast, rather than relying on subclasses to take care of this.
* Simplify setting self.add_prefix_space, ensure pre_tok exists
* Wrap in try-except to catch 'Custom PreTokenizer cannot be serialized'
862d1a346a/bindings/python/src/pre_tokenizers.rs (L672)
2025-01-09 17:46:50 +01:00
Cyril Vallez
46276f9a7f
Fix modular edge case + modular sorting order ( #35562 )
...
* look-ahead negation
* re add examples by default
* Fix the bug in topological sort
* Update create_dependency_mapping.py
* start adding test
* finalize test
* more tests
* style
* style
2025-01-09 17:17:52 +01:00
Amit Luhar
d3fe9fa3fe
PR for Issue #22694 : Fixed Training Evaluation table display for VSCode ( #35557 )
2025-01-09 15:05:47 +00:00
Pablo Montalvo
395b114bd1
Small fix rope kwargs ( #35589 )
...
* don't know why this keeps popping up?
* remove unused rope_kwargs
2025-01-09 15:40:36 +01:00
Yih-Dar
82dd6c14bb
Fix flaky SwitchTransformersModelTest::test_training_gradient ( #35587 )
...
* fix
* Update tests/models/switch_transformers/test_modeling_switch_transformers.py
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com >
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com >
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com >
2025-01-09 15:36:22 +01:00
Arthur
eb4579cf43
tokenizer train from iterator without pre_tokenizers (#35396 )
...
* fix if else issues
* add a test
* fix the test
* style
2025-01-09 15:34:43 +01:00
Mehant Kammakomati
320512df46
feat: add TP plan for granite ( #35573 )
...
Signed-off-by: Mehant Kammakomati <mehant.kammakomati2@ibm.com >
2025-01-09 15:25:55 +01:00
Saif Rehman Nasir
633da1b10e
[Idefics3] Move image features to same device as input embeds ( #35100 )
...
* [Idefics3] Move image features to same device as input embeds
* Update src/transformers/models/idefics3/modeling_idefics3.py
* make style
---------
Co-authored-by: Saif Rehman Nasir <shyshin@github.com >
Co-authored-by: Raushan Turganbay <raushan.turganbay@alumni.nu.edu.kz >
Co-authored-by: Raushan Turganbay <raushan@huggingface.co >
2025-01-09 14:25:36 +01:00
Jack Morris
832c6191ed
Add inputs_embeds param to ModernBertModel ( #35373 )
...
* update modular_modernbert -- add inputs_embeds param to ModernBertModel
* Fix implementation issues; extend to other classes; docstring
First of all, the inputs_embeds shouldn't fully replace `self.embeddings(input_ids)`, because this call also does layer normalization and dropout. So, now both input_ids and inputs_embeds is passed to the ModernBertEmbeddings, much like how BertEmbeddings is implemented.
I also added `inputs_embeds` to the docstring, and propagated the changes to the other model classes.
I also introduced an error if input_ids and input_embeds are both or neither provided.
Lastly, I fixed an issue with device being based solely on input_ids with attention_mask.
* Propagate inputs_embeds to ModernBertForMaskedLM correctly
Also reintroduce inputs_embeds test
---------
Co-authored-by: Tom Aarsen <Cubiegamedev@gmail.com >
2025-01-09 14:17:26 +01:00
Yih-Dar
1b2f942af7
Fix flaky test_batching_equivalence ( #35564 )
...
* yes!
* oh no!!!
* oh no!!!
* style
* oh no!!!
* oh no!!!
* oh no!!!
* oh no!!!
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com >
2025-01-09 14:00:08 +01:00
Chander G
4adc415b6d
Setup loss_type in config at model init time ( #34616 )
...
* setup loss_type in config at model init time
ensures no additional graph break introduced when torch.compile'ed
fixes #34615
Signed-off-by: ChanderG <mail@chandergovind.org >
* lookup loss mapping at init time instead of manual setup
Signed-off-by: ChanderG <mail@chandergovind.org >
* remove redundant lookup at loss_function time
* overwride losstype at init time
---------
Signed-off-by: ChanderG <mail@chandergovind.org >
Co-authored-by: Arthur Zucker <arthur.zucker@gmail.com >
2025-01-09 13:32:21 +01:00
Cyril Vallez
c8ab6ce6ce
Re-add missing __all__ for Cohere and Phi3 ( #35578 )
...
re-add missing __all__
2025-01-09 11:29:31 +01:00
Merve Noyan
487c31a21f
Minor fix in video text 2 text docs ( #35546 )
...
minor fix in docs
2025-01-09 11:20:36 +01:00
Cyril Vallez
965a2fb320
More model refactoring! ( #35359 )
...
* cohere
* style
* phi3
* style
* small fix
* small fix
* phi3 longrope
* oups
* Update rope (only for phi3 still)
* Update test_modeling_rope_utils.py
* Update modeling_phi3.py
* fix
* fix copies
* style
* Fix copied from bad renaming
2025-01-09 11:09:09 +01:00
Raushan Turganbay
137965ca7d
Don't show warning for inv_freq buffers ( #35255 )
...
dont show warning
2025-01-09 10:46:01 +01:00
Arthur
8cad65a698
Fix multi-gpu loss ( #35395 )
...
push to device
2025-01-09 10:14:31 +01:00
Arthur
2e2f8015c0
update code owners ( #35576 )
...
update
2025-01-09 09:55:41 +01:00
Ahmed Almaghz
a6256ec098
[i18n-ar] Translated file: docs/source/ar/tasks/multiple_choice.md into Arabic ( #35199 )
...
* إضافة الترجمة العربية: multiple_choice.md
* Update multiple_choice.md
* Update docs/source/ar/tasks/multiple_choice.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/tasks/multiple_choice.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/tasks/multiple_choice.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/tasks/multiple_choice.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/tasks/multiple_choice.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/tasks/multiple_choice.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/tasks/multiple_choice.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/tasks/multiple_choice.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/tasks/multiple_choice.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/tasks/multiple_choice.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/tasks/multiple_choice.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/tasks/multiple_choice.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/tasks/multiple_choice.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/tasks/multiple_choice.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/tasks/multiple_choice.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/tasks/multiple_choice.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/tasks/multiple_choice.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/tasks/multiple_choice.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/tasks/multiple_choice.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update _toctree.yml
* Add files via upload
* Update _toctree.yml
---------
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
2025-01-08 14:17:58 -08:00
nhamanasu
b32938aeee
Fix all output_dir in test_trainer.py to use tmp_dir ( #35266 )
...
* update codecarbon
* replace directly-specified-test-dirs with tmp_dir
* pass tmp_dir to all get_regression_trainer
* test_trainer.py: Use tmp_dir consistently for all output_dir arguments
* fix some with...as tmp_dir blocks
* reflect the comments to improve test_trainer.py
* refresh .gitignore
2025-01-08 19:44:39 +01:00
Joao Gante
76da6ca034
Pipeline: simple API for assisted generation ( #34504 )
...
Co-authored-by: Matt <Rocketknight1@users.noreply.github.com >
2025-01-08 17:08:02 +00:00
Arthur
3f483beab9
[PixtralLarge] Update Pixtral conversion script to support large format! ( #34801 )
...
* update conversion script
* update for bias again
* remove pdv
* use my dir
* Update how we initialize the tokenizer
* Convert in bfloat16
* Undo that one again
* fix config dump
* .to() was broken for BatchMixFeature
* quick debug breakpoint
* put the breakpoint in the right place
* Add a config flag for the multimodal projector bias
* Add a config flag for the multimodal projector bias
* Conversion script can load chat templates
* Indent config for comparison
* Stop clobbering the config
* Re-enable the config clobber
* Get rid of the config manual save - it has no effect!
* Handle adapter bias correctly
* Default vision transformer activation to silu
* Remove legacy processing path
* One commit with all the debug breakpoints before I delete them all, in case I need to revert
* Update conversion
* Remove vLLM debugging instrumentation
* Drop xformers
* Remove debug enumerates
* make fixup
* make fixup
* Break copied from in pixtral
* Propagate multimodal_projector_bias change
* Propagate multimodal_projector_bias change
* Remove debug device .to()
* Restore attention weights output
* Fix Pixtral test
* Drop image_seq_length
* Drop image_seq_length
* Put the legacy processing code back
* Add the bias option to the llava_next_video config
* Add the bias option to the llava_next_video config
* Make certain args required in converter
* Make certain args required in converter
* typo
* make fixup
* Reverting some dtype changes since it seems to work without them
---------
Co-authored-by: arthur@huggingface.co <arthur@ip-26-0-166-244.ec2.internal >
Co-authored-by: Matt <rocketknight1@gmail.com >
Co-authored-by: Matt <Rocketknight1@users.noreply.github.com >
2025-01-08 17:39:47 +01:00
DaNing An
4c2c12b3de
[docs] Remove Hiera from AUDIO MODELS in docs ( #35544 )
...
Remove Hiera from AUDIO MODELS
Hiera is a visual model and should not appear in audio model...
2025-01-08 16:33:21 +00:00
HERIUN
854dc7941b
ovewrite top_k when crate audio classification pipeline ( #35541 )
...
* ovewrite top_k when crate audio classification pipeline
* Update src/transformers/pipelines/audio_classification.py
---------
Co-authored-by: Matt <Rocketknight1@users.noreply.github.com >
2025-01-08 16:32:27 +00:00
Arthur
8c555ca3d7
add code owners ( #35528 )
...
* add co owners
* normal processing
* /src/transformers/models/*/*_modeling*
* Update CODEOWNERS
* Update CODEOWNERS
Co-authored-by: Yih-Dar <2521628+ydshieh@users.noreply.github.com >
* Update CODEOWNERS
Co-authored-by: Pavel Iakubovskii <qubvel@gmail.com >
* nit
* Apply suggestions from code review
Co-authored-by: Alvaro Moran <6949769+tengomucho@users.noreply.github.com >
Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com >
Co-authored-by: Matthew Douglas <38992547+matthewdouglas@users.noreply.github.com >
* Update CODEOWNERS
* rather put `@Rocketknight1`
---------
Co-authored-by: Yih-Dar <2521628+ydshieh@users.noreply.github.com >
Co-authored-by: Pavel Iakubovskii <qubvel@gmail.com >
Co-authored-by: Alvaro Moran <6949769+tengomucho@users.noreply.github.com >
Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com >
Co-authored-by: Matthew Douglas <38992547+matthewdouglas@users.noreply.github.com >
2025-01-08 17:14:44 +01:00
NielsRogge
8490d3159c
Add ViTPose ( #30530 )
...
* First draft
* Make fixup
* Make forward pass worké
* Improve code
* More improvements
* More improvements
* Make predictions match
* More improvements
* Improve image processor
* Fix model tests
* Add classic decoder
* Convert classic decoder
* Verify image processor
* Fix classic decoder logits
* Clean up
* Add post_process_pose_estimation
* Improve post_process_pose_estimation
* Use AutoBackbone
* Add support for MoE models
* Fix tests, improve num_experts%
* Improve variable names
* Make fixup
* More improvements
* Improve post_process_pose_estimation
* Compute centers and scales
* Improve postprocessing
* More improvements
* Fix ViTPoseBackbone tests
* Add docstrings, fix image processor tests
* Update index
* Use is_cv2_available
* Add model to toctree
* Add cv2 to doc tests
* Remove script
* Improve conversion script
* Add coco_to_pascal_voc
* Add box_to_center_and_scale to image_transforms
* Update tests
* Add integration test
* Fix merge
* Address comments
* Replace numpy by pytorch, improve docstrings
* Remove get_input_embeddings
* Address comments
* Move coco_to_pascal_voc
* Address comment
* Fix style
* Address comments
* Fix test
* Address comment
* Remove udp
* Remove comment
* [WIP] need to check if the numpy function is same as cv
* add scipy affine_transform
* Update src/transformers/models/vitpose/image_processing_vitpose.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com >
* refactor convert
* add output_shape
* add atol 5e-2
* Use hf_hub_download in conversion script
* make box_to_center more applicable
* skipt test_get_set_embedding
* fix to accept array and fix CI
* add co-contributor
* make it to tensor type output
* add torch
* change to torch tensor
* add more test
* minor change
* CI test change
* import torch should be above ImageProcessor
* make style
* try not use torch in def
* Update src/transformers/models/vitpose/image_processing_vitpose.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com >
* Update src/transformers/models/vitpose_backbone/configuration_vitpose_backbone.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com >
* Update src/transformers/models/vitpose_backbone/modeling_vitpose_backbone.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com >
* Update src/transformers/models/vitpose/modeling_vitpose.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com >
* fix
* fix
* add caution
* make more detail about dataset_index
* Update src/transformers/models/vitpose/modeling_vitpose.py
Co-authored-by: Sangbum Daniel Choi <34004152+SangbumChoi@users.noreply.github.com >
* Update src/transformers/models/vitpose/image_processing_vitpose.py
Co-authored-by: Sangbum Daniel Choi <34004152+SangbumChoi@users.noreply.github.com >
* add docs
* Update docs/source/en/model_doc/vitpose.md
* Update src/transformers/models/vitpose/configuration_vitpose.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com >
* Update src/transformers/__init__.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com >
* Revert "Update src/transformers/__init__.py"
This reverts commit 7ffa504450bb9dbccf9c7ea668441b98a1939d5c.
* change name
* Update src/transformers/models/vitpose/image_processing_vitpose.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com >
* Update tests/models/vitpose/test_modeling_vitpose.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com >
* Update docs/source/en/model_doc/vitpose.md
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com >
* Update src/transformers/models/vitpose/modeling_vitpose.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com >
* Update src/transformers/models/vitpose_backbone/modeling_vitpose_backbone.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com >
* Update src/transformers/models/vitpose/image_processing_vitpose.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com >
* move vitpose only function to image_processor
* raise valueerror when using timm backbone
* use out_indices
* Update src/transformers/models/vitpose/image_processing_vitpose.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com >
* remove camel-case of def flip_back
* rename vitposeEstimatorOutput
* Update src/transformers/models/vitpose_backbone/modeling_vitpose_backbone.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com >
* fix confused camelcase of MLP
* remove in-place logic
* clear scale description
* make consistent batch format
* docs update
* formatting docstring
* add batch tests
* test docs change
* Update src/transformers/models/vitpose/image_processing_vitpose.py
* Update src/transformers/models/vitpose/configuration_vitpose.py
* chagne ViT to Vit
* change to enable MoE
* make fix-copies
* Update docs/source/en/model_doc/vitpose.md
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com >
* extract udp
* add more described docs
* simple fix
* change to accept target_size
* make style
* Update src/transformers/models/vitpose/image_processing_vitpose.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com >
* Update src/transformers/models/vitpose/configuration_vitpose.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com >
* change to `verify_backbone_config_arguments`
* Update docs/source/en/model_doc/vitpose.md
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com >
* remove unnecessary copy
* make config immutable
* enable gradient checkpointing
* update inappropriate docstring
* linting docs
* split function for visibility
* make style
* check isinstances
* change to acceptable use_pretrained_backbone
* make style
* remove copy in docs
* Update src/transformers/models/vitpose_backbone/modeling_vitpose_backbone.py
Co-authored-by: Pavel Iakubovskii <qubvel@gmail.com >
* Update docs/source/en/model_doc/vitpose.md
Co-authored-by: Pavel Iakubovskii <qubvel@gmail.com >
* Update src/transformers/models/vitpose/modeling_vitpose.py
Co-authored-by: Pavel Iakubovskii <qubvel@gmail.com >
* simple fix + make style
* change input config of activation function to string
* Update docs/source/en/model_doc/vitpose.md
Co-authored-by: Pavel Iakubovskii <qubvel@gmail.com >
* tmp docs
* delete index.md
* make fix-copies
* simple fix
* change conversion to sam2/mllama style
* Update src/transformers/models/vitpose/image_processing_vitpose.py
Co-authored-by: Pavel Iakubovskii <qubvel@gmail.com >
* Update src/transformers/models/vitpose/image_processing_vitpose.py
Co-authored-by: Pavel Iakubovskii <qubvel@gmail.com >
* refactor convert
* add supervision
* Update src/transformers/models/vitpose_backbone/modeling_vitpose_backbone.py
Co-authored-by: Pavel Iakubovskii <qubvel@gmail.com >
* remove reduntant def
* seperate code block for visualization
* add validation for num_moe
* final commit
* add labels
* [run-slow] vitpose, vitpose_backbone
* Update src/transformers/models/vitpose/convert_vitpose_to_hf.py
Co-authored-by: Pavel Iakubovskii <qubvel@gmail.com >
* enable all conversion
* final commit
* [run-slow] vitpose, vitpose_backbone
* ruff check --fix
* [run-slow] vitpose, vitpose_backbone
* rename split module
* [run-slow] vitpose, vitpose_backbone
* fix pos_embed
* Simplify init
* Revert "fix pos_embed"
This reverts commit 2c56a4806e30bc9b5753b142fa04b913306c54ff.
* refactor single loop
* allow flag to enable custom model
* efficiency of MoE to not use unused experts
* make style
* Fix range -> arange to avoid warning
* Revert MOE router, a new one does not work
* Fix postprocessing a bit (labels)
* Fix type hint
* Fix docs snippets
* Fix links to checkpoints
* Fix checkpoints in tests
* Fix test
* Add image to docs
---------
Co-authored-by: Niels Rogge <nielsrogge@nielss-mbp.home >
Co-authored-by: Niels Rogge <nielsrogge@Nielss-MacBook-Pro.local >
Co-authored-by: sangbumchoi <danielsejong55@gmail.com >
Co-authored-by: Sangbum Daniel Choi <34004152+SangbumChoi@users.noreply.github.com >
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com >
Co-authored-by: Pavel Iakubovskii <qubvel@gmail.com >
2025-01-08 16:02:14 +00:00
Minho Shim
4349a0e401
fix: Qwen2-VL generate with inputs_embeds ( #35466 )
...
* fix: Qwen2-VL generate with inputs_embeds
* change: optional input_ids in get_rope_index
2025-01-08 16:36:03 +01:00
Sean (Seok-Won) Yi
88e18b3c63
Update doc for metric_for_best_model when save_strategy="best". ( #35389 )
...
* Updated docstring for _determine_best_metric.
* Updated docstring for metric_for_best_model.
* Added test case for save strategy.
* Updated incorrect test case.
* Changed eval_strategy to match save_strategy.
* Separated test cases for metric.
* Allow load_best_model when save_strategy == "best".
* Updated docstring for metric_for_best_model.
2025-01-08 16:32:35 +01:00
jp
29e74b7cbc
Add: num_additional_image_tokens to models ( #35052 )
...
* Add: num_additional_image_tokens to models
* docs: update docstring for num_additional_image_tokens in configuration files
* Add num_additional_image_tokens to LlavaNextVideo model and update feature selection logic
* revert
* Fix: adjust num_image_tokens calculation in LlavaProcessor
* Remove num_additional_image_tokens initialization from configuration files
* Fix test error
* revert
* Fix: adjust num_image_tokens calculation in LlavaNextVideoProcessor
* fix conflict
* Fix: adjust num_image_tokens calculation in VideoLlavaProcessor
* make style
---------
Co-authored-by: Raushan Turganbay <raushan@huggingface.co >
2025-01-08 16:20:01 +01:00
Pavel Iakubovskii
657bb14f98
Enable auto task for timm models in pipeline ( #35531 )
...
* Enable auto task for timm models
* Add pipeline test
2025-01-08 15:14:17 +00:00
Yichen Yan
1a6c1d3a9a
Bump torch requirement to >= 2 ( #35479 )
...
Bump torch requirement, follow-up of #35358
2025-01-08 15:59:32 +01:00
Pavel Iakubovskii
59e5b3f01b
Timm wrapper label names ( #35553 )
...
* Add timm wrapper label names mapping
* Add index to classification pipeline
* Revert adding index for pipelines
* Add custom model check for loading timm labels
* Add tests for labels
* [run-slow] timm_wrapper
* Add note regarding label2id mapping
2025-01-08 14:09:46 +00:00
Matt
f1639ea51d
Update missing model error message ( #35370 )
...
* Update missing model error message
* Update missing model error message
* Update missing model error message
* Fix capitalization
2025-01-08 15:05:06 +01:00
Jade Choghari
bd39b0627b
Update doc and default value of TextNetImageProcessor ( #35563 )
...
update doc and default value
2025-01-08 13:47:52 +00:00
Yoni Gozlan
651cfb400f
Add support for modular with fast image processors ( #35379 )
...
* Add support for modular with fast image processors
* fix order and remove copied from
* add comment for "image_processing*_fast"
2025-01-08 08:37:57 -05:00
Joao Gante
430d3d43a5
[Docs] links to logits-processor-zoo ( #35552 )
...
links to logits-processor-zoo
2025-01-08 13:36:30 +00:00
Jacky Lee
3c1895aa65
Fix Qwen2VL processor to handle odd number of frames ( #35431 )
...
* fix: processing odd number of frames
* feat: add test case
* update: test one frame
* feat: support custom patch size
* fix: test with videos
* revert: change on patch repeat
* fix: much wow
* update: fixups
* fixup pls
* ruff fixup
* fix typo at least
2025-01-08 13:49:00 +01:00
