Avishai Elmakies
78b2929c05
Sdpa dino v2 ( #33403 )
...
* add sdpa to dinov2
* fixup
* add dinov2 to sdpa doc
* update doc order
* [run-slow] dinov2
* common to eager
* [run-slow] dinov2
* update attn implementation in common
* update test_modeling_dinov2 to have mask_ration, num_masks and mask_length similar to vit
* [run-slow] dinov2
---------
Co-authored-by: Avishai Elmakies <avishai.elma@cs.huji.ac.il >
2024-09-21 01:58:00 +01:00
amyeroberts
e71bf70e33
Pixtral update example checkpoint ( #33633 )
...
* Update pixtral example checkpoint
* Fix typo
2024-09-21 01:01:16 +01:00
Mayank Mishra
e472e077c2
Granitemoe ( #33207 )
...
* first commit
* drop tokenizer
* drop tokenizer
* drop tokenizer
* drop convert
* granite
* drop tokenization test
* mup
* fix
* reformat
* reformat
* reformat
* fix docs
* stop checking for checkpoint
* update support
* attention multiplier
* update model
* tiny drop
* saibo drop
* skip test
* fix test
* fix test
* drop
* drop useless imports
* update docs
* drop flash function
* copied from
* drop pretraining tp
* drop pretraining tp
* drop pretraining tp
* drop unused import
* drop code path
* change name
* softmax scale
* head dim
* drop legacy cache
* rename params
* cleanup
* fix copies
* comments
* add back legacy cache
* multipliers
* multipliers
* multipliers
* text fix
* fix copies
* merge
* multipliers
* attention multiplier
* drop unused imports
* add granitemoe
* add decoration
* remove moe from sequenceclassification
* fix test
* fix
* fix
* fix
* move rope?
* merge
* drop bias
* drop bias
* Update src/transformers/models/granite/configuration_granite.py
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com >
* fix
* Update src/transformers/models/granite/modeling_granite.py
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com >
* fix
* fix
* fix
* fix
* drop
* drop
* fix
* fix
* cleanup
* cleanup
* fix
* fix granite tests
* fp32 test
* fix
* drop jitter
* fix
* rename
* rename
* fix config
* add gen test
---------
Co-authored-by: Yikang Shen <yikang.shn@gmail.com >
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com >
2024-09-21 01:43:50 +02:00
Omar Salman
653eb40425
Add sdpa for BioGpt ( #33592 )
...
* Add sdpa for BioGpt
* Updates
* Add the docs
* [run_slow] biogpt
* Use the copy mechanism to ensure consistency
* [run_slow] biogpt
2024-09-20 14:27:32 +01:00
Yoni Gozlan
f111d5b783
Uniformize kwargs for Paligemma processor and update docs ( #33571 )
...
* Uniformize paligemma processor
* nit
2024-09-19 14:14:06 -04:00
Joao Gante
80b774eb29
Cache: don't show warning in forward passes when past_key_values is None ( #33541 )
2024-09-19 12:02:46 +01:00
Yoach Lacombe
5af7d41e49
Codec integration ( #33565 )
...
* clean mimi commit
* some nits suggestions from Arthur
* make fixup
* rename repo id + change readme
* Update docs/source/en/model_doc/mimi.md
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com >
* add flaky flag to batching equivalence due to audio_codes failing sometimes
---------
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com >
2024-09-18 19:23:44 +02:00
Aymeric Roucher
e6d9f39dd7
Decorator for easier tool building ( #33439 )
...
* Decorator for tool building
2024-09-18 11:07:51 +02:00
Yoni Gozlan
d8500cd229
Uniformize kwargs for Pixtral processor ( #33521 )
...
* add uniformized pixtral and kwargs
* update doc
* fix _validate_images_text_input_order
* nit
2024-09-17 14:44:27 -04:00
Antoine Dussolle
763548427d
Add explicit example for RAG chat templating ( #33503 )
...
* Add explicit example for RAG chat templating
* Add Tip box and reformulate
Co-authored-by: Matt <Rocketknight1@users.noreply.github.com >
---------
Co-authored-by: Matt <Rocketknight1@users.noreply.github.com >
2024-09-17 16:08:05 +01:00
Max Buckley
ac5a0556f1
Update chameleon.md — fix runtime type error ( #33494 )
...
Update chameleon.md
Fix error
RuntimeError: Input type (float) and bias type (c10::BFloat16) should be the same
2024-09-17 13:32:49 +02:00
Ahmed Almaghz
c2d05897bf
[i18n-ar] Add File : docs/source/ar/_toctree.yml ( #32696 )
...
* Update ar lang build_documentation.yml
* Update ar lang build_pr_documentation.yml
* Update docs/source/ar/pipeline_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/pipeline_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/pipeline_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/pipeline_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/pipeline_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/pipeline_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/pipeline_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/pipeline_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/pipeline_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/pipeline_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/pipeline_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/pipeline_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/pipeline_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/pipeline_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/pipeline_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/pipeline_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/pipeline_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/pipeline_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/pipeline_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/pipeline_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/pipeline_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/pipeline_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/pipeline_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/pipeline_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/pipeline_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/autoclass_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/autoclass_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/autoclass_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/autoclass_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/autoclass_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/autoclass_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/autoclass_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/autoclass_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/autoclass_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/autoclass_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/autoclass_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/autoclass_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/autoclass_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/autoclass_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/training.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/training.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/training.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/training.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/training.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/training.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/training.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/training.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/training.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/training.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/training.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/training.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/training.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/training.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/training.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/training.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/training.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/training.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/training.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/training.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/run_scripts.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/run_scripts.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/run_scripts.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/run_scripts.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/run_scripts.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/run_scripts.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/run_scripts.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/accelerate.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/accelerate.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/accelerate.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/accelerate.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/accelerate.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/accelerate.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Create _config.py
* Update _toctree.yml
* Update _toctree.yml
* Update docs/source/ar/peft.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/peft.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/peft.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/peft.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/peft.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/peft.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/peft.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/peft.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/peft.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update _toctree.yml
* Update docs/source/ar/model_sharing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/model_sharing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/model_sharing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/model_sharing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/model_sharing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/model_sharing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/model_sharing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/model_sharing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/model_sharing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/model_sharing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/model_sharing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/model_sharing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/model_sharing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/model_sharing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/model_sharing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/model_sharing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/model_sharing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/model_sharing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/model_sharing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/model_sharing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/conversations.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/conversations.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/conversations.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/conversations.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/conversations.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/conversations.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/conversations.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/conversations.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/conversations.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/conversations.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/conversations.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/conversations.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/conversations.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/conversations.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/conversations.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/llm_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/llm_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/llm_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/llm_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/llm_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/llm_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/llm_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/llm_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/llm_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/llm_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/llm_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/llm_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/llm_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/llm_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/llm_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/llm_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/llm_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/llm_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/llm_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/llm_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/llm_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update llm_tutorial.md
* Update _toctree.yml
* Update autoclass_tutorial.md
* Update autoclass_tutorial.md
* Update preprocessing.md
* Update glossary.md
* Update run_scripts.md
* Update run_scripts.md
* Update run_scripts.md
---------
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
2024-09-16 10:02:03 -07:00
Sergio Paniego Blanco
c7a91f5adf
Agents, supercharged - Multi-agents, External tools, and more docs typo fixed (#33478 )
...
* Typo fixed in Agents, supercharged
2024-09-16 18:52:27 +02:00
Merve Noyan
ce62a41880
Add keypoint-detection task guide ( #33274 )
...
---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
2024-09-16 13:08:31 +02:00
Arthur
8bd2b1e8c2
Add support for Pixtral ( #33449 )
...
* initial commit
* gloups
* updates
* work
* weights match
* nits
* nits
* updates to support the tokenizer :)
* updates
* Pixtral processor (#33454 )
* rough outline
* Add in image break and end tokens
* Fix
* Udo some formatting changes
* Set patch_size default
* Fix
* Fix token expansion
* nit in conversion script
* Fix image token list creation
* done
* add expected results
* Process list of list of images (#33465 )
* updates
* working image and processor
* this is the expected format
* some fixes
* push current updated
* working mult images!
* add a small integration test
* Uodate configuration docstring
* Formatting
* Config docstring fix
* simplify model test
* fixup modeling and etests
* Return BatchMixFeature in image processor
* fix some copies
* update
* nits
* Update model docstring
* Apply suggestions from code review
* Fix up
* updates
* revert modeling changes
* update
* update
* fix load safe
* addd liscence
* update
* use pixel_values as required by the model
* skip some tests and refactor
* Add pixtral image processing tests (#33476 )
* Image processing tests
* Add processing tests
* woops
* defaults reflect pixtral image processor
* fixup post merge
* images -> pixel values
* oups sorry Mr docbuilder
* isort
* fix
* fix processor tests
* small fixes
* nit
* update
* last nits
* oups this was really breaking!
* nits
* is composition needs to be true
---------
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com >
2024-09-14 12:28:39 +02:00
Sergio Paniego Blanco
e39b6c1c7c
Corrected Agents and tools documentation links typos ( #33471 )
...
* Corrected agents task link typo
* Corrected chat templating link
* Corrected chat templating link 2
2024-09-13 17:15:20 +02:00
Fanli Lin
a05ce550bf
[docs] refine the doc for train with a script ( #33423 )
...
* add xpu note
* add one more case
* add more
* Update docs/source/en/run_scripts.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
2024-09-12 10:16:12 -07:00
Raushan Turganbay
2f611d30d9
Qwen2-VL: clean-up and add more tests ( #33354 )
...
* clean-up on qwen2-vl and add generation tests
* add video tests
* Update tests/models/qwen2_vl/test_processing_qwen2_vl.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com >
* fix and add better tests
* Update src/transformers/models/qwen2_vl/image_processing_qwen2_vl.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com >
* update docs and address comments
* Update docs/source/en/model_doc/qwen2_vl.md
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com >
* Update docs/source/en/model_doc/qwen2_vl.md
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com >
* update
* remove size at all
---------
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com >
2024-09-12 18:24:04 +02:00
Sergio Paniego Blanco
516ee6adc2
Fix incomplete sentence in Zero-shot object detection documentation ( #33430 )
...
Rephrase sentence in zero-shot object detection docs
2024-09-12 11:25:44 +02:00
Michael Currin
e0ff4321d1
Docs - update formatting of llama3 model card ( #33438 )
...
update formatting of llama3 content
2024-09-12 11:24:56 +02:00
Fanli Lin
cea9ec086a
[docs] add the missing tokenizer when pushing models to huggingface hub ( #33428 )
...
* add tokenizer
* typo
2024-09-11 09:56:55 -07:00
Fanli Lin
c403441339
[docs] add the missing huggingface hub username ( #33431 )
...
* add username
* update username
* add username
2024-09-11 09:56:40 -07:00
Guang Yang
f38590dade
Make StaticCache configurable at model construct time ( #32830 )
...
* Make StaticCache configurable at model construct time
* integrations import structure
* add new doc file to toc
---------
Co-authored-by: Guang Yang <guangyang@fb.com >
Co-authored-by: Joao Gante <joao@huggingface.co >
2024-09-10 16:35:57 +01:00
Alazar
96429e74a8
Add support for GGUF Phi-3 ( #31844 )
...
* Update docs for GGUF supported models
* Add tensor mappings and define class GGUFPhi3Converter
* Fix tokenizer
* Working version
* Attempt to fix some CI failures
* Run ruff format
* Add vocab, merges, decoder methods like LlamaConverter
* Resolve conflicts since Qwen2Moe was added to gguf
- I missed one place when resolving conflict
- I also made a mistake with tests_ggml.py and now has been fixed to reflect
its master version.
2024-09-10 13:32:38 +02:00
Nilay Bhatnagar
eedd21b9e7
Fixed Majority of the Typos in transformers[en] Documentation ( #33350 )
...
* Fixed typo: insted to instead
* Fixed typo: relase to release
* Fixed typo: nighlty to nightly
* Fixed typos: versatible, benchamarks, becnhmark to versatile, benchmark, benchmarks
* Fixed typo in comment: quantizd to quantized
* Fixed typo: architecutre to architecture
* Fixed typo: contibution to contribution
* Fixed typo: Presequities to Prerequisites
* Fixed typo: faste to faster
* Fixed typo: extendeding to extending
* Fixed typo: segmetantion_maps to segmentation_maps
* Fixed typo: Alternativelly to Alternatively
* Fixed incorrectly defined variable: output to output_disabled
* Fixed typo in library name: tranformers.onnx to transformers.onnx
* Fixed missing import: import tensorflow as tf
* Fixed incorrectly defined variable: token_tensor to tokens_tensor
* Fixed missing import: import torch
* Fixed incorrectly defined variable and typo: uromaize to uromanize
* Fixed incorrectly defined variable and typo: uromaize to uromanize
* Fixed typo in function args: numpy.ndarry to numpy.ndarray
* Fixed Inconsistent Library Name: Torchscript to TorchScript
* Fixed Inconsistent Class Name: OneformerProcessor to OneFormerProcessor
* Fixed Inconsistent Class Named Typo: TFLNetForMultipleChoice to TFXLNetForMultipleChoice
* Fixed Inconsistent Library Name Typo: Pytorch to PyTorch
* Fixed Inconsistent Function Name Typo: captureWarning to captureWarnings
* Fixed Inconsistent Library Name Typo: Pytorch to PyTorch
* Fixed Inconsistent Class Name Typo: TrainingArgument to TrainingArguments
* Fixed Inconsistent Model Name Typo: Swin2R to Swin2SR
* Fixed Inconsistent Model Name Typo: EART to BERT
* Fixed Inconsistent Library Name Typo: TensorFLow to TensorFlow
* Fixed Broken Link for Speech Emotion Classification with Wav2Vec2
* Fixed minor missing word Typo
* Fixed minor missing word Typo
* Fixed minor missing word Typo
* Fixed minor missing word Typo
* Fixed minor missing word Typo
* Fixed minor missing word Typo
* Fixed minor missing word Typo
* Fixed minor missing word Typo
* Fixed Punctuation: Two commas
* Fixed Punctuation: No Space between XLM-R and is
* Fixed Punctuation: No Space between [~accelerate.Accelerator.backward] and method
* Added backticks to display model.fit() in codeblock
* Added backticks to display openai-community/gpt2 in codeblock
* Fixed Minor Typo: will to with
* Fixed Minor Typo: is to are
* Fixed Minor Typo: in to on
* Fixed Minor Typo: inhibits to exhibits
* Fixed Minor Typo: they need to it needs
* Fixed Minor Typo: cast the load the checkpoints To load the checkpoints
* Fixed Inconsistent Class Name Typo: TFCamembertForCasualLM to TFCamembertForCausalLM
* Fixed typo in attribute name: outputs.last_hidden_states to outputs.last_hidden_state
* Added missing verbosity level: fatal
* Fixed Minor Typo: take To takes
* Fixed Minor Typo: heuristic To heuristics
* Fixed Minor Typo: setting To settings
* Fixed Minor Typo: Content To Contents
* Fixed Minor Typo: millions To million
* Fixed Minor Typo: difference To differences
* Fixed Minor Typo: while extract To which extracts
* Fixed Minor Typo: Hereby To Here
* Fixed Minor Typo: addition To additional
* Fixed Minor Typo: supports To supported
* Fixed Minor Typo: so that benchmark results TO as a consequence, benchmark
* Fixed Minor Typo: a To an
* Fixed Minor Typo: a To an
* Fixed Minor Typo: Chain-of-though To Chain-of-thought
2024-09-09 10:47:24 +02:00
Aymeric Roucher
489cbfd6d3
Add visit webpage tool ( #33353 )
...
* Add VisitWebpageTool
2024-09-09 10:32:42 +02:00
Wing Lian
62aecd85ff
schedulefree optimizers ( #30079 )
...
* schedulefree optimizers
* fix train instead of eval for optimizer
* fixes and update docs
* chore: lint
* add tests and drop overly-verbose _32bit suffix
* chore: lint
* fix for docs
* fix code review issues
* use duck-typing to avoid per-optimizer patches
* fixup style
* fixup style
* warn if incorrect accelerate version with schedule free
Co-authored-by: Aman Gupta Karmani <aman@tmm1.net >
---------
Co-authored-by: Aman Karmani <aman@tmm1.net >
2024-09-09 09:51:39 +02:00
Nicholas Broad
66bc4def95
add sdpa mbart ( #32033 )
...
* add sdpa mbart
useful for donut
* update sdpa docs
* formatting
* add self._use_sdpa in mbartencoder
* use self.config to check attn
* retrigger checks
* [run-slow] mbart
2024-09-06 17:31:24 -07:00
Daniel Lok
a70286f827
Update author for QLorA/PEFT community notebook ( #33338 )
...
update author
Signed-off-by: Daniel Lok <daniel.lok@databricks.com >
2024-09-06 22:50:26 +02:00
Matt
d7b04ea14d
Fix Prefill docs ( #33352 )
...
last -> final
2024-09-06 17:57:54 +01:00
Ita Zaporozhets
e48e5f1f13
Support reading tiktoken tokenizer.model file ( #31656 )
...
* use existing TikTokenConverter to read tiktoken tokenizer.model file
* del test file
* create titktoken integration file
* adding tiktoken llama test
* ALTNATIVE IMPLEMENTATION: supports llama 405B
* fix one char
* remove redundant line
* small fix
* rm unused import
* flag for converting from tiktokeng
* remove unneeded file
* ruff
* remove llamatiktokenconverter, stick to general converter
* tiktoken support v2
* update test
* remove stale changes
* udpate doc
* protect import
* use is_protobuf_available
* add templateprocessor in tiktokenconverter
* reverting templateprocessor from tiktoken support
* update test
* add require_tiktoken
* dev-ci
* trigger build
* trigger build again
* dev-ci
* [build-ci-image] tiktoken
* dev-ci
* dev-ci
* dev-ci
* dev-ci
* change tiktoken file name
* feedback review
* feedback rev
* applying feedback, removing tiktoken converters
* conform test
* adding docs for review
* add doc file for review
* add doc file for review
* add doc file for review
* support loading model without config.json file
* Revert "support loading model without config.json file"
This reverts commit 2753602e51c34cef2f184eb11f36d2ad1b02babb.
* remove dev var
* updating docs
* safely import protobuf
* fix protobuf import error
* fix protobuf import error
* trying isort to fix ruff error
* fix ruff error
* try to fix ruff again
* try to fix ruff again
* try to fix ruff again
* doc table of contents
* add fix for consistency.dockerfile torchaudio
* ruff
* applying feedback
* minor typo
* merging with push-ci-image
* clean up imports
* revert dockerfile consistency
2024-09-06 14:24:02 +02:00
Joao Gante
2b789f27f3
Docs: add more cross-references to the KV cache docs ( #33323 )
...
* add more cross-references
* nit
* import guard
* more import guards
* nit
* Update src/transformers/generation/configuration_utils.py
2024-09-06 10:22:00 +01:00
Daniel Lok
5792c459ed
Add a community notebook for fine-tuning with QLoRA, PEFT, and MLflow ( #33319 )
...
add notebook for finetuning with mlflow
Signed-off-by: Daniel Lok <daniel.lok@databricks.com >
2024-09-06 09:35:01 +02:00
Vladislav Bronzov
5d11de4a2f
Add Qwen2Moe GGUF loading support ( #33264 )
...
* update gguf doc, config and tensor mapping
* add qwen2moe architecture support, GGUFQwen2MoeConverter and q4 unit tests
* apply code style fixes
* reformat files
* assign GGUFQwen2Converter to qwen2_moe
2024-09-05 17:42:03 +02:00
Niklas Muennighoff
03164ba14e
Add paper link ( #33305 )
2024-09-05 15:49:28 +02:00
Raushan Turganbay
43df47d8e7
Llava Onevision: add model ( #32673 )
...
* working version
* fix copies
* update
* tests
* update docs
* codestyle
* add more tests
* add returns for docs
* clean up
* Update src/transformers/models/llava_onevision/processing_llava_onevision.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com >
* updates
* codestyle
* style
* shouldn't be reversed
* [run-slow] llava_onevision
* [run-slow] llava_onevision
* add pooling in videos
* [run-slow] llava_onevision
* num-logits-to-keep
* [run-slow] llava_onevision
* [run-slow] llava_onevision
* Update tests/test_modeling_common.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com >
* video matched orig impl
* fix tests
* chat template was modified
* Update docs/source/en/model_doc/llava_onevision.md
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com >
* add morer info in the doc page
---------
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com >
2024-09-05 14:43:20 +05:00
Aymeric Roucher
cfd92c64f5
Add new documentation page for advanced agent usage ( #33265 )
...
* Add new documentation page for advanced agent usage
2024-09-04 18:19:54 +02:00
Matt
01c8c6c419
Add a warning to the chat template docs about the tool_calls format ( #33277 )
...
* Add a warning to the chat template docs
* Add a warning to the chat template docs
* Add a warning to the chat template docs
2024-09-04 17:13:34 +01:00
Raushan Turganbay
ebbe8d8014
Cache docs: update ( #32929 )
...
* some changes
* more updates
* fix cache copy
* nits
* nits
* add tests
2024-09-04 15:05:31 +05:00
Niklas Muennighoff
ecd61c6286
Add OLMoE ( #32406 )
...
* Add OLMoE
* Add OLMoE
* Updates
* Make norm optional; add keys
* Add output
* Add
* Fix dtype
* Fix eos config
* Update
* Add OLMoE
* Fix OLMoE path
* Format
* Format
* Rmv copy statement
* Rmv copy statement
* Format
* Add copies
* Cp rotary
* Fix aming
* Fix naming
* Update RoPE integration; num_logits_to_keep; Add copy statements
* Add eps to config
* Format
* Add aux loss
* Adapt router_aux_loss_coef
* Update md
* Adapt
* adapt tests
2024-09-03 18:43:12 +02:00
Omar Salman
03c12d0d63
Add sdpa support for Albert ( #32092 )
...
* Add sdpa support for Albert
* [run_slow] albert
* Add benchmarks and PR suggestion
* Fix quality
* Fix
* [run_slow] albert
2024-09-03 14:01:00 +01:00
Matt
0d86727354
Update chat template docs to remove Blenderbot ( #33254 )
...
* Update docs to remove obsolete Blenderbot
* Remove another reference to Blenderbot
2024-09-03 12:18:04 +01:00
Isotr0py
edeca4387c
🚨 Support dequantization for most GGML types ( #32625 )
...
* use gguf internal dequantize
* add Q5_0 test
* add iq1 test
* add remained test
* remove duplicated test
* update docs
* add gguf version limit
* make style
* update gguf import catch
* revert vocab_size patch
* make style
* use GGUF_MIN_VERSION everywhere
2024-09-03 12:58:14 +02:00
Sergio Paniego Blanco
28952248b1
Fixed typo repeated word in DETR docs ( #33250 )
2024-09-02 17:19:18 +02:00
Matt
52a0213755
Add assistant prefill for chat templates and TextGenerationPipeline ( #33198 )
...
* Add assistant prefill to chat templates
* Add assistant prefill to pipeline
* Add assistant prefill to pipeline
* Tweak another test that ended in assistant message
* Update tests that ended in assistant messages
* Update tests that ended in assistant messages
* Replace assistant_prefill with continue_final_message
* Allow passing continue_final_message to pipeline
* Small fixup
* Add continue_final_message as a pipeline kwarg
* Update docstrings
* Move repos to hf-internal-testing!
* Update src/transformers/tokenization_utils_base.py
Co-authored-by: Lysandre Debut <hi@lysand.re >
* Add explanatory comment
* make fixup
* Update chat templating docs to explain continue_last_message
---------
Co-authored-by: Lysandre Debut <hi@lysand.re >
2024-09-02 13:23:47 +01:00
Aymeric Roucher
1ca9ff5c91
Add duckduckgo search tool ( #32882 )
...
* Add duckduckgo search tool
2024-09-02 09:56:20 +02:00
Merve Noyan
2e3f8f7474
Add video text to text docs ( #33164 )
...
---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
2024-09-01 12:06:31 +03:00
Yijun Lee
db70426854
🌐 [i18n-KO] Translated llm_optims.md to Korean ( #32325 )
...
* docs: ko: llm_optims.md
* feat: nmt draft
* fix toc title
* fix: manual edits
* Update docs/source/ko/llm_optims.md
Co-authored-by: Jiwook Han <33192762+mreraser@users.noreply.github.com >
* Update docs/source/ko/llm_optims.md
Co-authored-by: Jiwook Han <33192762+mreraser@users.noreply.github.com >
* Update docs/source/ko/llm_optims.md
Co-authored-by: Jiwook Han <33192762+mreraser@users.noreply.github.com >
* Update docs/source/ko/llm_optims.md
Co-authored-by: Jiwook Han <33192762+mreraser@users.noreply.github.com >
* Update docs/source/ko/llm_optims.md
Co-authored-by: Jiwook Han <33192762+mreraser@users.noreply.github.com >
* Update docs/source/ko/llm_optims.md
Co-authored-by: Jiwook Han <33192762+mreraser@users.noreply.github.com >
* Update docs/source/ko/llm_optims.md
Co-authored-by: Jiwook Han <33192762+mreraser@users.noreply.github.com >
* Update docs/source/ko/llm_optims.md
Co-authored-by: Jiwook Han <33192762+mreraser@users.noreply.github.com >
* Update docs/source/ko/llm_optims.md
Co-authored-by: Jiwook Han <33192762+mreraser@users.noreply.github.com >
* Update docs/source/ko/llm_optims.md
Co-authored-by: HyunJi Shin <74661937+shinhyunji36@users.noreply.github.com >
* Update docs/source/ko/llm_optims.md
Co-authored-by: HyunJi Shin <74661937+shinhyunji36@users.noreply.github.com >
* Update llm_optims.md
* fix: resolve suggestions
* fix: resolve suggestions
* Apply suggestions from code review
fix: resolve suggestions
Co-authored-by: Jiwook Han <33192762+mreraser@users.noreply.github.com >
---------
Co-authored-by: Jiwook Han <33192762+mreraser@users.noreply.github.com >
Co-authored-by: HyunJi Shin <74661937+shinhyunji36@users.noreply.github.com >
2024-08-30 09:52:41 -07:00
Aymeric Roucher
c79bfc71b8
Create local Transformers Engine ( #33218 )
...
* Create local Transformers Engine
2024-08-30 18:22:27 +02:00
Gerben van V
5129671290
Add a static cache that offloads to the CPU or other device ( #32161 )
...
* Add a static cache that offloads to the CPU or other device
* Fix PR comments, add unit-tests
2024-08-29 11:51:09 +02:00
