nhamanasu
3d213b57fe
skip Fuyu from test_generate ( #35246 )
...
* skip Fuyu from test_generate
* make fixup, quality, repo-consistency
2024-12-13 10:12:49 +01:00
Nadav Timor
e3ee49fcfb
Refactoring AssistedCandidateGenerator for Improved Modularity and Reusability ( #35009 )
...
* move `TestAssistedCandidateGeneratorDifferentTokenizers` into a new testing file
* refactor
* NOTHING. add space to rerun github actions tests
* remove it...
* NOTHING. add space to rerun github actions tests
* remove it...
* replace: `self.prev_tokens` -> `self.prev_assistant_ids`
* NOTHING. rerun CI tests
* remove it
* introduce `self.prev_target_ids_len`
* fix style
* fix style
---------
Co-authored-by: Jonathan Mamou <jonathan.mamou@intel.com >
2024-12-12 15:47:05 +01:00
Aymeric Roucher
9ad4c93536
Add Aria ( #34157 )
...
* Add Aria
---------
Co-authored-by: Cyril Vallez <cyril.vallez@gmail.com >
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com >
2024-12-06 12:17:34 +01:00
Jonathan Mamou
e27465c801
Adaptive dynamic number of speculative tokens ( #34156 )
...
* initial commit
* update strategy
* add tradeoff FPR TPR with cost
* all probs
* fix
* fix
* fix style
* Update src/transformers/generation/configuration_utils.py
shorter docstring
Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com >
* import guard
* fix style
* add is_sklearn_available condition
* vectorizing to flatten the for-loop
* fix style
* disable adaptation for UAG
* update doc
* add TestAssistedCandidateGeneratorUpdateStrategy
* fix style
* protect import
* fix style
---------
Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com >
2024-12-05 17:07:33 +01:00
Yih-Dar
b0a51e5cff
Fix flaky Hub CI (test_trainer.py) ( #35062 )
...
* fix
* Update src/transformers/testing_utils.py
Co-authored-by: Lucain <lucainp@gmail.com >
* fix
* fix
* fix
* fix
* fix
* fix
* fix
* fix
* check
* check
* check
* check
* check
* check
* Update src/transformers/testing_utils.py
Co-authored-by: Lucain <lucainp@gmail.com >
* Update src/transformers/testing_utils.py
Co-authored-by: Lucain <lucainp@gmail.com >
* check
* check
* check
* Final space
* Final adjustment
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com >
Co-authored-by: Lucain <lucainp@gmail.com >
2024-12-05 17:02:27 +01:00
Raushan Turganbay
5e8c1d713d
Offloaded cache: fix generate ( #34921 )
...
* fix cache impl
* require_torch_gpu
* fix mamba
* fix copies
2024-11-28 15:05:56 +01:00
xinpengzz
44af935ec5
Refine the code of Universal Assisted Generation ( #34823 )
...
* removed the useless attritbutes
* add configs for window size
* fixed the wrong kwargs
* added docstring
2024-11-28 15:04:24 +01:00
jiqing-feng
a464afbe2a
fix static cache data type miss-match ( #34799 )
...
* fix gptj data type missmatch
Signed-off-by: jiqing-feng <jiqing.feng@intel.com >
* add low precision static cache tests
Signed-off-by: jiqing-feng <jiqing.feng@intel.com >
* fix format
Signed-off-by: jiqing-feng <jiqing.feng@intel.com >
* fix low-precision static cache tests
* fix format
Signed-off-by: jiqing-feng <jiqing.feng@intel.com >
* avoid config change
Signed-off-by: jiqing-feng <jiqing.feng@intel.com >
* change data type convert in cache copy
Signed-off-by: jiqing-feng <jiqing.feng@intel.com >
* fix comment
Signed-off-by: jiqing-feng <jiqing.feng@intel.com >
* cast key value after k v out
Signed-off-by: jiqing-feng <jiqing.feng@intel.com >
---------
Signed-off-by: jiqing-feng <jiqing.feng@intel.com >
2024-11-25 16:59:38 +01:00
Raushan Turganbay
c1a8520419
Cache: init empty cache when use_cache ( #34274 )
...
* fix
* fix tests
* fix copies
* add docs
* Revert "add docs"
This reverts commit 32d35634f12ba02781d2ebdee0c8dcfbe992a7b9.
* qwen move deltas
* mllama can potentiall fullgraph compile
* enable mllama compile and fix tests
* remove mllama fixes
2024-11-25 10:11:33 +01:00
Nadav Timor
42b36d7395
Speculative decoding: Test the target distribution (to prevent issues like #32867 ) ( #34553 )
...
* Update test_utils.py
* formatting
* Update test_utils.py
* formatting
* formatting
* Update test_utils.py
* formatting
* Update test_utils.py
* formatting
* format
* comments at standard positions
2024-11-22 16:02:37 +01:00
Raushan Turganbay
28fb02fc05
VLMs: enable generation tests - last batch ( #34484 )
...
* add tests for 3 more vlms
* fix fuyu back
* skip test
2024-11-21 11:00:22 +01:00
Raushan Turganbay
9470d65324
Fix low memory beam search ( #34746 )
...
* fix
* higher max positions in tests
2024-11-20 07:46:35 +01:00
Arthur
4bff54f921
Gemma capping ( #34282 )
...
* softcapping
* soft cap before the mask
* style
* ...
* super nit
* update
* fixes
* update
* small issue with modular
* fix modular imports
* update
* fixup
* simplify a hell lot
* simplify cleaning imports
* finish fixing
* update our design
* nits
* use a deprecation cycle
* updates
* Fix modular (recursive deps need to always be computed after merges!)
* push
* fix
* update
* fix modular order
* make fix-copies
* updates
* update
* ?
* don't compile for now
* ?
* fix some stuff
* donc!
* fix copies
* update
* fixup
* ?
* fix two tests
* fix?
* for now, don't use head info
* eager when output attentoin and sdpa or flash as it's the simplest behaviour (for our tests as well :))
* fix-copies
* revert sdpa check
* Apply suggestions from code review
Co-authored-by: Cyril Vallez <cyril.vallez@huggingface.co >
* rebase, fix-copies and push
* add a slow integration test
* update the test
* fix left padding issue
* fix test
* remove duplicate scaling
* quality
* add a small test and make sure it works
* 2b
---------
Co-authored-by: Cyril Vallez <cyril.vallez@gmail.com >
Co-authored-by: Cyril Vallez <cyril.vallez@huggingface.co >
2024-11-19 13:52:38 +01:00
Arthur
54739a320e
Self-speculation (Layer-Skip Llama) ( #34240 )
...
* 😅
* early exit (#34244 )
* mvp
* docs and tests
* a few fixes
* no shared cache
* Apply suggestions from code review
Co-authored-by: Mostafa Elhoushi <m.elhoushi@ieee.org >
* docs
* make fix-copies
* cohere fix
* [test all]
* [test all] consistent model code copies
* [test all] make fix-copies :D
* Apply suggestions from code review
Co-authored-by: Pedro Cuenca <pedro@huggingface.co >
Co-authored-by: Mostafa Elhoushi <m.elhoushi@ieee.org >
* Update src/transformers/generation/candidate_generator.py
* Update src/transformers/generation/configuration_utils.py
Co-authored-by: Pedro Cuenca <pedro@huggingface.co >
* [test all] don't use a stand-alone attribute; fix test
---------
Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com >
Co-authored-by: Joao Gante <joao@huggingface.co >
Co-authored-by: Mostafa Elhoushi <m.elhoushi@ieee.org >
Co-authored-by: Pedro Cuenca <pedro@huggingface.co >
2024-11-19 12:20:07 +00:00
Yih-Dar
f2d5dfbab2
Remove @slow for test_eager_matches_sdpa_inference ( #34558 )
...
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com >
2024-11-05 16:10:42 +01:00
Raushan Turganbay
4cc0813e28
BLIP: enable generation tests ( #34174 )
...
* blip2 tests
* instructblips
* copies
* fix slow tests
* fix
* uncomment this
* clean up after rebase
* should be model main input
* fix overwritten tests
* oops len should be multiple of frame number
* style
* fix some tests
2024-11-01 08:54:48 +01:00
Yih-Dar
114dd812dd
make test_eager_matches_sdpa_inference less flaky ( #34512 )
...
* try
* try
* try
* try
* try
* try
* update
* update
* update
* update
* update
* update
* update
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com >
2024-10-31 18:34:00 +01:00
Joao Gante
4ca004eac6
Qwen2VL: skip base input_ids-inputs_embeds equivalence check ( #34535 )
...
it has complex inputs_embeds computation
2024-10-31 15:42:13 +00:00
Joao Gante
8a734ea2c3
Tests: move generate tests to the right mixin and delete redundant tests ( #34464 )
...
* tmp commit
* tmp commit
* cull overwrites of deleted tests
* typo
* more specific docstring
* make fixup
* parameterize at the top?
* correction
* more deletions :D
* tmp commit
* for VLMs too
* fix _check_outputs
* test nit
* make fixup
* fix another flaky
* test_generate_from_inputs_embeds -- handle missing attention mask
2024-10-30 10:59:08 +00:00
Raushan Turganbay
63ca6d9771
Fix CI ( #34458 )
...
* fix
* fix mistral
2024-10-29 08:26:04 +01:00
Raushan Turganbay
808d6c50f8
Generation: fix test ( #34369 )
...
* fix test
* fix copies
2024-10-29 07:57:10 +01:00
Joao Gante
186b8dc190
Tests: upgrade test_eager_matches_sdpa_generate ( #34386 )
2024-10-25 11:55:07 +01:00
Joao Gante
b0f0c61899
Add SynthID (watermerking by Google DeepMind) ( #34350 )
...
* Add SynthIDTextWatermarkLogitsProcessor
* esolving comments.
* Resolving comments.
* esolving commits,
* Improving SynthIDWatermark tests.
* switch to PT version
* detector as pretrained model + style
* update training + style
* rebase
* Update logits_process.py
* Improving SynthIDWatermark tests.
* Shift detector training to wikitext negatives and stabilize with lower learning rate.
* Clean up.
* in for 7B
* cleanup
* upport python 3.8.
* README and final cleanup.
* HF Hub upload and initiaze.
* Update requirements for synthid_text.
* Adding SynthIDTextWatermarkDetector.
* Detector testing.
* Documentation changes.
* Copyrights fix.
* Fix detector api.
* ironing out errors
* ironing out errors
* training checks
* make fixup and make fix-copies
* docstrings and add to docs
* copyright
* BC
* test docstrings
* move import
* protect type hints
* top level imports
* watermarking example
* direct imports
* tpr fpr meaning
* process_kwargs
* SynthIDTextWatermarkingConfig docstring
* assert -> exception
* example updates
* no immutable dict (cant be serialized)
* pack fn
* einsum equivalent
* import order
* fix test on gpu
* add detector example
---------
Co-authored-by: Sumedh Ghaisas <sumedhg@google.com >
Co-authored-by: Marc Sun <marc@huggingface.co >
Co-authored-by: sumedhghaisas2 <138781311+sumedhghaisas2@users.noreply.github.com >
Co-authored-by: raushan <raushan@huggingface.co >
2024-10-23 21:18:52 +01:00
Raushan Turganbay
ca541bd4f4
Generation tests: don't rely on main input name ( #34228 )
...
* don't rely on main input name
* update
2024-10-21 10:00:14 +02:00
alpertunga-bile
98bad9c6d6
[fix] fix token healing tests and usage errors ( #33931 )
...
* auto-gptq requirement is removed & model is changed & tokenizer pad token is assigned
* values func is changed with extensions & sequence key value bug is fixed
* map key value check is added in ExtensionsTree
* empty trimmed_ids bug is fixed
* tail_id IndexError is fixed
* empty trimmed_ids bug fix is updated for failed test
* too much specific case for specific tokenizer is removed
* input_ids check is updated
* require auto-gptq import is removed
* key error check is changed with empty list check
* empty input_ids check is added
* empty trimmed_ids fix is checked with numel function
* usage change comments are added
* test changes are commented
* comment style and quality bugs are fixed
* test comment style and quality bug is fixed
2024-10-16 14:22:55 +02:00
Yoach Lacombe
9ba021ea75
Moshi integration ( #33624 )
...
* clean mimi commit
* some nits suggestions from Arthur
* make fixup
* first moshi WIP
* converting weights working + configuration + generation configuration
* finalize converting script - still missing tokenizer and FE and processor
* fix saving model w/o default config
* working generation
* use GenerationMixin instead of inheriting
* add delay pattern mask
* fix right order: moshi codes then user codes
* unconditional inputs + generation config
* get rid of MoshiGenerationConfig
* blank user inputs
* update convert script:fix conversion, add tokenizer, feature extractor and bf16
* add and correct Auto classes
* update modeling code, configuration and tests
* make fixup
* fix some copies
* WIP: add integration tests
* add dummy objects
* propose better readiblity and code organisation
* update tokenization tests
* update docstrigns, eval and modeling
* add .md
* make fixup
* add MoshiForConditionalGeneration to ignore Auto
* revert mimi changes
* re
* further fix
* Update moshi.md
* correct md formating
* move prepare causal mask to class
* fix copies
* fix depth decoder causal
* fix and correct some tests
* make style and update .md
* correct config checkpoitn
* Update tests/models/moshi/test_tokenization_moshi.py
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com >
* Update tests/models/moshi/test_tokenization_moshi.py
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com >
* make style
* Update src/transformers/models/moshi/__init__.py
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com >
* fixup
* change firm in copyrights
* udpate config with nested dict
* replace einsum
* make style
* change split to True
* add back splt=False
* remove tests in convert
* Update tests/models/moshi/test_modeling_moshi.py
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com >
* add default config repo + add model to FA2 docstrings
* remove logits float
* fix some tokenization tests and ignore some others
* make style tokenization tests
* update modeling with sliding window + update modeling tests
* [run-slow] moshi
* remove prepare for generation frol CausalLM
* isort
* remove copied from
* ignore offload tests
* update causal mask and prepare 4D mask aligned with recent changes
* further test refine + add back prepare_inputs_for_generation for depth decoder
* correct conditional use of prepare mask
* update slow integration tests
* fix multi-device forward
* remove previous solution to device_map
* save_load is flaky
* fix generate multi-devices
* fix device
* move tensor to int
---------
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com >
Co-authored-by: Marc Sun <marc@huggingface.co >
2024-10-16 11:21:49 +02:00
Raushan Turganbay
23874f5948
Idefics: enable generation tests ( #34062 )
...
* add idefics
* conflicts after merging main
* enable tests but need to fix some
* fix tests
* no print
* fix/skip some slow tests
* continue not skip
* rebasing broken smth, this is the fix
2024-10-15 11:17:14 +02:00
Yih-Dar
80bee7b114
Avoid many test failures for LlavaNextVideoForConditionalGeneration ( #34070 )
...
* skip
* [run-slow] llava_next_video
* skip
* [run-slow] video_llava, llava_next_video
* skip
* [run-slow] llava_next_video
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com >
2024-10-11 17:41:50 +02:00
Joao Gante
37ac078535
Generate: move prepare_inputs_for_generation in encoder-decoder llms ( #34048 )
2024-10-11 16:11:18 +01:00
Lucain
1c66be8062
Fix PushToHubMixin when pusing to a PR revision ( #34090 )
2024-10-11 15:06:15 +02:00
Matthew Hoffman
70b07d97cf
Default synced_gpus to True when using FullyShardedDataParallel ( #33483 )
...
* Default synced_gpus to True when using FullyShardedDataParallel
Fixes #30228
Related:
* https://github.com/pytorch/pytorch/issues/100069
* https://github.com/pytorch/pytorch/issues/123962
Similar to DeepSpeed ZeRO Stage 3, when using FSDP with multiple GPUs and differently sized data per rank, the ranks reach different synchronization points at the same time, leading to deadlock
To avoid this, we can automatically set synced_gpus to True if we detect that a PreTrainedModel is being managed by FSDP using _is_fsdp_managed_module, which was added in 2.0.0 for torch.compile: https://github.com/pytorch/pytorch/blob/v2.0.0/torch/distributed/fsdp/_dynamo_utils.py
* Remove test file
* ruff formatting
* ruff format
* Update copyright year
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com >
* Add test for FSDP-wrapped model generation
Before #33483 , these tests would have hung for 10 minutes before crashing due to a timeout error
* Ruff format
* Move argparse import
* Remove barrier
I think this might cause more problems if one of the workers was killed
* Move import into function to decrease load time
https://github.com/huggingface/transformers/pull/33483#discussion_r1787972735
* Add test for accelerate and Trainer
https://github.com/huggingface/transformers/pull/33483#discussion_r1790309675
* Refactor imports
* Ruff format
* Use nullcontext
---------
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com >
2024-10-10 14:09:04 -04:00
Daniel Korat
fb0c6b521d
Universal Assisted Generation: Assisted generation with any assistant model (by Intel Labs) ( #33383 )
...
* Update candidate_generator.py
* Update utils.py
* add lookbehind params to _get_candidate_generator
* make fixup
* add unit tests
* fix failing tests
* add docstrings
* fix docstrings; remove non-optimized AnyTokenizer
* added any tokenizer generation correctness test
* make fixup
* fix assertion syntax
* PR review fixes
* address additional PR comments
* fix tests
* remove stropping criteria arg
* make fixup
* add AssistantConfig
* fix prev_tokens branching
* pass tokenizers through `generate()`kwargs
* fix lookbehind values; tokenizer params WIP
* fixup
* AssistantConfig
* remove AssistantConfig; apply PR suggestions
* restructure tests
* fixup
* fix assistant_tokenizer arg validation
* fixup
* fix tests in TestAssistedCandidateGeneratorDifferentTokenizers
* fix class docstring
* PR suggestions
* doc
* doc update and improvements to `_validate_assistant()`
---------
Co-authored-by: mosheber <moshe.berchansky@intel.com >
2024-10-10 14:41:53 +02:00
Joao Gante
295a90cb40
Generate: remove most decoder-only LLMs prepare_inputs_for_generation ( #33870 )
2024-10-09 12:15:48 +01:00
Joao Gante
38f9f10dd9
Cache: revert DynamicCache init for BC ( #33861 )
...
* tmp commit
* tmp commit
* make fixup
* missing removal
* fix condition
* fix end-to-end compilation
* if -> elif
* BC
* BC
* use @deprecate_kwarg("num_hidden_layers", version="4.47.0")
* wups the import
* 🥴
---------
Co-authored-by: Arthur Zucker <arthur.zucker@gmail.com >
2024-10-04 22:47:08 +02:00
pglorio
f319ba16fa
Add Zamba ( #30950 )
...
* Update index.md
* Rebase
* Rebase
* Updates from make fixup
* Update zamba.md
* Batched inference
* Update
* Fix tests
* Fix tests
* Fix tests
* Fix tests
* Update docs/source/en/model_doc/zamba.md
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com >
* Update docs/source/en/model_doc/zamba.md
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com >
* Update configuration_zamba.py
* Update src/transformers/models/zamba/modeling_zamba.py
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com >
* Update src/transformers/models/zamba/modeling_zamba.py
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com >
* Update src/transformers/models/zamba/modeling_zamba.py
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com >
* Update src/transformers/models/zamba/modeling_zamba.py
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com >
* Update modeling_zamba.py
* Update modeling_zamba.py
* Update modeling_zamba.py
* Update configuration_zamba.py
* Update modeling_zamba.py
* Update modeling_zamba.py
* Merge branch 'main' of https://github.com/Zyphra/transformers_zamba
* Update ZambaForCausalLM
* Update ZambaForCausalLM
* Describe diffs with original mamba layer
* Moved mamba init into `_init_weights`
* Update index.md
* Rebase
* Rebase
* Updates from make fixup
* Update zamba.md
* Batched inference
* Update
* Fix tests
* Fix tests
* Fix tests
* Fix tests
* Update docs/source/en/model_doc/zamba.md
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com >
* Update docs/source/en/model_doc/zamba.md
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com >
* Update configuration_zamba.py
* Update src/transformers/models/zamba/modeling_zamba.py
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com >
* Update src/transformers/models/zamba/modeling_zamba.py
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com >
* Update src/transformers/models/zamba/modeling_zamba.py
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com >
* Update src/transformers/models/zamba/modeling_zamba.py
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com >
* Update modeling_zamba.py
* Update modeling_zamba.py
* Update modeling_zamba.py
* Update configuration_zamba.py
* Update modeling_zamba.py
* Update modeling_zamba.py
* Merge branch 'main' of https://github.com/Zyphra/transformers_zamba
* Update ZambaForCausalLM
* Moved mamba init into `_init_weights`
* Update ZambaForCausalLM
* Describe diffs with original mamba layer
* make fixup fixes
* quality test fixes
* Fix Zamba model path
* circleci fixes
* circleci fixes
* circleci fixes
* circleci fixes
* circleci fixes
* circleci fixes
* circleci fixes
* circleci fixes
* circleci fixes
* Update
* circleci fixes
* fix zamba test from merge
* fix ValueError for disabling mamba kernels
* add HF copyright
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com >
* shared_transf --> shared_transformer
* Update src/transformers/models/zamba/modeling_zamba.py
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com >
* Update src/transformers/models/zamba/modeling_zamba.py
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com >
* Fixes
* Move attention head dim to config
* Fix circle/ci tests
* Update modeling_zamba.py
* apply GenerationMixin inheritance change from upstream
* apply import ordering
* update needed transformers version for zamba
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com >
* add contribution author
* add @slow to avoid CI
* Update src/transformers/models/zamba/modeling_zamba.py
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com >
* Define attention_hidden_size
* Added doc for attention_head_size
* trigger CI
* Fix doc of attention_hidden_size
* [run-slow] zamba
* Fixed shared layer logic, swapped up<->gate in mlp
* shared_transformer -> shared_transf
* reformat HybridLayer __init__
* fix docstrings in zamba config
* added definition of _get_input_ids_and_config
* fixed formatting of _get_input_ids_and_config
---------
Co-authored-by: root <root@node-4.us-southcentral1-a.compute.internal >
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com >
Co-authored-by: root <root@node-1.us-southcentral1-a.compute.internal >
Co-authored-by: Quentin Anthony <qganthony@yahoo.com >
2024-10-04 22:28:05 +02:00
Joao Gante
d29738f5b4
Generate tests: modality-agnostic input preparation ( #33685 )
2024-10-03 14:01:24 +01:00
Marc Sun
cac4a4876b
[Quantization] Switch to optimum-quanto ( #31732 )
...
* switch to optimum-quanto rebase squach
* fix import check
* again
* test try-except
* style
2024-10-02 15:14:34 +02:00
Arthur
19d58d31f1
Add MLLama ( #33703 )
...
* current changes
* nit
* Add cross_attenttion_mask to processor
* multi-image fixed
* Add cross_attenttion_mask to processor
* cross attn works in all cases
* WIP refactoring function for image processor
* WIP refactoring image processor functions
* Refactor preprocess to use global loops instead of list nested list comps
* Docstrings
* Add channels unification
* fix dtype issues
* Update docsrings and format
* Consistent max_image_tiles
* current script
* updates
* Add convert to rgb
* Add image processor tests
* updates!
* update
* god damn it I am dumb sometimes
* Precompute aspect ratios
* now this works, full match
* fix 😉
* nits
* style
* fix model and conversion
* nit
* nit
* kinda works
* hack for sdpa non-contiguous bias
* nits here and there
* latest c hanges
* merge?
* run forward
* Add aspect_ratio_mask
* vision attention mask
* update script and config variable names
* nit
* nits
* be able to load
* style
* nits
* there
* nits
* make forward run
* small update
* enable generation multi-turn
* nit
* nit
* Clean up a bit for errors and typos
* A bit more constant fixes
* 90B keys and shapes match
* Fix for 11B model
* Fixup, remove debug part
* Docs
* Make max_aspect_ratio_id to be minimal
* Update image processing code to match new implementation
* Adjust conversion for final checkpoint state
* Change dim in repeat_interleave (accordig to meta code)
* tmp fix for num_tiles
* Fix for conversion (gate<->up, q/k_proj rope permute)
* nits
* codestyle
* Vision encoder fixes
* pass cross attn mask further
* Refactor aspect ratio mask
* Disable text-only generation
* Fix cross attention layers order, remove q/k norm rotation for cross atention layers
* Refactor gated position embeddings
* fix bugs but needs test with new weights
* rope scaling should be llama3
* Fix rope scaling name
* Remove debug for linear layer
* fix copies
* Make mask prepare private func
* Remove linear patch embed
* Make precomputed embeddings as nn.Embedding module
* MllamaPrecomputedAspectRatioEmbedding with config init
* Remove unused self.output_dim
* nit, intermediate layers
* Rename ln and pos_embed
* vision_chunk_size -> image_size
* return_intermediate -> intermediate_layers_indices
* vision_input_dim -> hidden_size
* Fix copied from statements
* fix most tests
* Fix more copied from
* layer_id->layer_idx
* Comment
* Fix tests for processor
* Copied from for _prepare_4d_causal_attention_mask_with_cache_position
* Style fix
* Add MllamaForCausalLM
* WIP fixing tests
* Remove duplicated layers
* Remove dummy file
* Fix style
* Fix consistency
* Fix some TODOs
* fix language_model instantiation, add docstring
* Move docstring, remove todos for precomputed embeds (we cannot init them properly)
* Add initial docstrings
* Fix
* fix some tests
* lets skip these
* nits, remove print, style
* Add one more copied from
* Improve test message
* Make validate func private
* Fix dummy objects
* Refactor `data_format` a bit + add comment
* typos/nits
Co-authored-by: Pablo Montalvo <39954772+molbap@users.noreply.github.com >
* fix dummy objects and imports
* Add chat template config json
* remove num_kv_heads from vision attention
* fix
* move some commits and add more tests
* fix test
* Remove `update_key_name` from modeling utils
* remove num-kv-heads again
* some prelimiary docs
* Update chat template + tests
* nit, conversion script max_num_tiles from params
* Fix warning for text-only generation
* Update conversion script for instruct models
* Update chat template in converstion + test
* add tests for CausalLM model
* model_max_length, avoid null chat_template
* Refactor conversion script
* Fix forward
* Fix integration tests
* Refactor vision config + docs
* Fix default
* Refactor text config
* Doc fixes
* Remove unused args, fix docs example
* Squashed commit of the following:
commit b51ce5a2efffbecdefbf6fc92ee87372ec9d8830
Author: qubvel <qubvel@gmail.com >
Date: Wed Sep 18 13:39:15 2024 +0000
Move model + add output hidden states and output attentions
* Fix num_channels
* Add mllama text and mllama vision models
* Fixing repo consistency
* Style fix
* Fixing repo consistency
* Fixing unused config params
* Fix failed tests after refactoring
* hidden_activation -> hidden_act for text mlp
* Remove from_pretrained from sub-configs
* Apply suggestions from code review
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com >
* Update src/transformers/models/mllama/convert_mllama_weights_to_hf.py
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com >
* Reuse lambda in conversion script
* Remove run.py
* Update docs/source/en/model_doc/mllama.md
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com >
* Update src/transformers/models/mllama/processing_mllama.py
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com >
* Remove unused LlamaTokenizerFast
* Fix logging
* Refactor gating
* Remove cycle for collecting intermediate states
* Refactor text-only check, add integration test for text-only
* Revert from pretrained to configs
* Fix example
* Add auto `bos_token` adding in processor
* Fix tips
* Update src/transformers/models/auto/tokenization_auto.py
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com >
* Enable supports_gradient_checkpointing model flag
* add eager/sdpa options
* don't skip attn tests and bring back GC skips (did i really remove those?)
* Fix signature, but get error with None gradient
* Fix output attention tests
* Disable GC back
* Change no split modules
* Fix dropout
* Style
* Add Mllama to sdpa list
* Add post init for vision model
* Refine config for MllamaForCausalLMModelTest and skipped tests for CausalLM model
* if skipped, say it, don't pass
* Clean vision tester config
* Doc for args
* Update tests/models/mllama/test_modeling_mllama.py
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com >
* Add cross_attention_mask to test
* typehint
* Remove todo
* Enable gradient checkpointing
* Docstring
* Style
* Fixing and skipping some tests for new cache
* Mark flaky test
* Skip `test_sdpa_can_compile_dynamic` test
* Fixing some offload tests
* Add direct GenerationMixin inheritance
* Remove unused code
* Add initializer_range to vision config
* update the test to make sure we show if split
* fix gc?
* Fix repo consistency
* Undo modeling utils debug changes
* Fix link
* mllama -> Mllama
* [mllama] -> [Mllama]
* Enable compile test for CausalLM model (text-only)
* Fix TextModel prefix
* Update doc
* Docs for forward, type hints, and vision model prefix
* make sure to reset
* fix init
* small script refactor and styling
* nit
* updates!
* some nits
* Interpolate embeddings for 560 size and update integration tests
* nit
* does not suppor static cache!
* update
* fix
* nit2
* this?
* Fix conversion
* Style
* 4x memory improvement with image cache AFAIK
* Token decorator for tests
* Skip failing tests
* update processor errors
* fix split issues
* style
* weird
* style
* fix failing tests
* update
* nit fixing the whisper tests
* fix path
* update
---------
Co-authored-by: raushan <raushan@huggingface.co >
Co-authored-by: pavel <ubuntu@ip-10-90-0-11.ec2.internal >
Co-authored-by: qubvel <qubvel@gmail.com >
Co-authored-by: Pablo Montalvo <39954772+molbap@users.noreply.github.com >
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com >
Co-authored-by: Pedro Cuenca <pedro@huggingface.co >
2024-09-25 19:56:25 +02:00
Jonathan Mamou
52daf4ec76
🚨 🚨 Setting default behavior of assisted decoding ( #33657 )
2024-09-25 09:39:09 +01:00
Joao Gante
a7734238ff
Generation tests: update imagegpt input name, remove unused functions ( #33663 )
2024-09-24 16:40:48 +01:00
Joao Gante
e15687fffe
Generation: deprecate PreTrainedModel inheriting from GenerationMixin ( #33203 )
2024-09-23 18:28:36 +01:00
Yih-Dar
077b552f07
Fix some missing tests in circleci ( #33559 )
...
* fix
* fix
* fix
* fix
* skip
* skip more
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com >
2024-09-20 20:58:51 +02:00
Duc-Viet Hoang
dc8b6eaeee
Fix contrastive search to correctly handle input with padding ( #33507 )
...
* fix: handle padding in contrastive search for decoder-only models
* fix: handle padding in contrastive search for encoder-decoder models
* tests: move padding contrastive test to test_util, add t5 test
* fix: handle if model_kwargs["decoder_attention_mask"] is None
* refactor: improve padding input contrastive search generation tests
* chore: _ranking_fast to use LongTensor for cosine_matrix_mask
2024-09-20 16:52:08 +01:00
Yih-Dar
31caf0b95f
Fix missing test in torch_job ( #33593 )
...
fix missing tests
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com >
2024-09-20 17:16:44 +02:00
Joao Gante
2fdb5e74cc
VLM generate: tests can't generate image/video tokens ( #33623 )
2024-09-20 15:43:27 +01:00
Joao Gante
266d0a6375
Generate: remove flakyness in test_generate_from_inputs_embeds_decoder_only ( #33602 )
...
almost zero is not zero
2024-09-20 14:50:42 +02:00
Vladislav Bronzov
162056a3f4
change sequence_bias type of SequenceBiasLogitsProcessor to list, add… ( #33375 )
...
* change sequence_bias type of SequenceBiasLogitsProcessor tp list, add config tests for all processors
* fix format
* small fix for all_token_bias_pairs_are_valid internal func
* small typo fix in description
* improve test impl, some SequenceBiasLogitsProcessor refactoring
2024-09-19 17:35:44 +01:00
Raushan Turganbay
d7975a5874
VLMs: enable generation tests ( #33533 )
...
* add tests
* fix whisper
* update
* nit
* add qwen2-vl
* more updates!
* better this way
* fix this one
* fix more tests
* fix final tests, hope so
* fix led
* Update tests/generation/test_utils.py
Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com >
* pr comments
* not pass pixels and extra for low-mem tests, very flaky because of visio tower
---------
Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com >
2024-09-19 12:04:24 +02:00
Marc Sun
6cc4dfe3f1
Fix the initialization of the cache when we have multi gpu ( #33303 )
...
* init cache multi-gpu
* Update src/transformers/generation/utils.py
Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com >
* switch to execution device map
* naming more consistant
* fix
* mutually exclusive device
* added an integration example
* remove useless check
* suggestion from joao + typing
* fix couple of typo and add test
* revert check
---------
Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com >
2024-09-13 15:06:08 +02:00
Jonathan Mamou
7a51cbc65f
Dynamic number of speculative tokens in order to accelerate speculative decoding ( #33258 )
...
* optimal Speculation Lookahead based on probability
* update peer finished condition
* add support to do_sample True
* add stopping criteria
* gitignore
* add print
* remove prints
* minor
* minor
* git ignore
* adding test to stopping ConfidenceCriteria
* doc + format
* add doc
* Update .gitignore
* update docstring and default value of assistant_confidence_threshold
* add docstring
* Update src/transformers/generation/configuration_utils.py
implicit default value (None)
Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com >
* style fix
---------
Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com >
2024-09-11 14:22:28 +02:00
