* [PEFT] Support low_cpu_mem_usage for PEFT loading
PEFT added support for low_cpu_mem_usage=True when loading adapters in
https://github.com/huggingface/peft/pull/1961. This feature is now
available when installing PEFT v0.13.0. With this PR, this option is
also supported when loading PEFT adapters directly into transformers
models.
Additionally, with this PR,
https://github.com/huggingface/diffusers/pull/9510 will be unblocked,
which implements this option in diffusers.
* Fix typo
* fix beam indices in token_timestamps
* fix attention_mask in FA2
* correct translation example with the right example
* correct how somes tests are using outputs + correct num_frames
* fix shortform batch prev cond tests
* make fix-copies
* make fix-copies
* take care of shifting beam indices
* [run-slow] whisper
* [run-slow] whisper
* add unit tests for splinter_tokenizer
* add unit test for splinter tokenizer, pass in the question_token to be saved on save_pretrained called
* remove unused import
* remove vocab_splinter.txt, add Copied from, use fmt:on and fmt:off to prevent autoformatting on long lines
* remove all the spaces
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
---------
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
* Use all state dict keys when checking if root module is initialized.
* Apply style corrections
* Add comment explaining change.
* Change comment phrasing.
* Update an keyerror on _save_check_point prevent confusion of missing metric keys
* Update grammar error and case sensitive.
Co-authored-by: Marc Sun <57196510+SunMarc@users.noreply.github.com>
* adding update KeyError on _evaluate function to align with _save_checkpoint function
---------
Co-authored-by: Marc Sun <57196510+SunMarc@users.noreply.github.com>
* When we set self.dt_proj.bias = None, it removes the bias parameter from the model. When we later tried to assign a tensor to self.dt_proj.bias, it caused a TypeError because PyTorch expects a Parameter object.
* When we set self.dt_proj.bias = None, it removes the bias parameter from the model. When we later tried to assign a tensor to self.dt_proj.bias, it caused a TypeError because PyTorch expects a Parameter object.
* When we set self.dt_proj.bias = None, it removes the bias parameter from the model. When we later tried to assign a tensor to self.dt_proj.bias, it caused a TypeError because PyTorch expects a Parameter object.
* Trainer - deprecate tokenizer for processing_class
* Extend chage across Seq2Seq trainer and docs
* Add tests
* Update to FutureWarning and add deprecation version
* add support for custom inputs and batched inputs in ProcessorTesterMixin
* Fix batch_size behavior ProcessorTesterMixin
* Change format prepare inputs batched
* Remove override test pixtral processor
* Remove unnecessary tests and cleanup after new prepare_inputs functions
* Fix instructBlipVideo image processor
* fix(copy): fixup copy
* fix(deformable_detr): move weight initialization to the right place
* fix(grounding_dino): move weight initialization to the right place
* fix(rt_detr): move weight initialization to the right place
* [run-slow] deformable_detr, grounding_dino, rt_detr
* Remove max_new_tokens arg
* Add ASR pipeline to testing
* make fixup
* Factor the output test out into a util
* Full error reporting
* Full error reporting
* Update src/transformers/pipelines/automatic_speech_recognition.py
Co-authored-by: Lysandre Debut <hi@lysand.re>
* Small comment
---------
Co-authored-by: Lysandre Debut <hi@lysand.re>
* Add include_loss_for_metrics
* Fix styling
* Initialize inputs and losses to avoid AttributeError
* Ruff styling
* Refactor compute_metrics and update EvalPrediction
* Change Naming
* Added include_for_metrics to group both args
* Fix style
* Change warnings to logger
Co-authored-by: Marc Sun <57196510+SunMarc@users.noreply.github.com>
---------
Co-authored-by: Marc Sun <57196510+SunMarc@users.noreply.github.com>
* fix(m2m_100): skip dropout in eval for flash_attn
* fix(misc): skip dropout in eval for flash attn various models
* chore(m2m_100): copy flash attn from bart
* chore: run make fix-copies
* [run-slow] bart, m2m_100
* refactor image features selection
* break line
* remove whitespace
* add pr comments: include projection and rename function
* make fix-copies
* fix get_image_feature in vip llava
* Fix Mamba slow path bug with dtype mismatch.
* Update test_modeling_mamba.py
* Improve style.
* Fix issue with cache position of dtype mismatch test.
* Change test for slow path.
* Revert changes.
* Switch to buggy code and add test to catch it.
* Fix the dtype mismatch bug and add test code to verify it.
* Fix minor bug with test.
* Fix incorrect dtype of model output.
* Fix incorrect dtype of cache.
* Fix incorrect dtype of ssm cache.
* Fix incorrect dtype of conv state.
* Remove assertion for ssm state.
* Add assertion for conv state dtype.
* Fix all issues with dtype mismatch test.
