wwwbai
3033509327
Translate attention.md into Chinese ( #34716 )
...
* try
* tryagain
* tryagggain
* translated
* translated2
* Update docs/source/zh/attention.md
Co-authored-by: Huazhong Ji <hzji210@gmail.com >
---------
Co-authored-by: Huazhong Ji <hzji210@gmail.com >
2024-11-19 10:03:12 -08:00
Merve Noyan
befbbf2f98
Added image-text-to-text pipeline to task guide ( #34783 )
...
* Added image-text-to-text pipeline to task guide
* Update docs/source/en/tasks/image_text_to_text.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update docs/source/en/tasks/image_text_to_text.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update docs/source/en/tasks/image_text_to_text.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update docs/source/en/tasks/image_text_to_text.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Merge codeblocks
---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
2024-11-19 09:49:10 -08:00
Yoni Gozlan
eedc113914
Add Image Processor Fast Deformable DETR ( #34353 )
...
* add deformable detr image processor fast
* add fast processor to doc
* fix copies
* nit docstring
* Add tests gpu/cpu and fix docstrings
* fix docstring
* import changes from detr
* fix imports
* rebase and fix
* fix input data format change in detr and rtdetr fast
2024-11-19 11:18:58 -05:00
David Zhang
427b62ed1a
Fix post process function called in the instance segmentation example of mask2former ( #34588 )
...
* Fix post process function called in the instance segmentation example of mask2former
* fix description and additional notes for post_process_instance_segmentation of maskformers
* remove white space in maskformers post_process_instance_segmentation doc
* change image.size[::-1] to height and width for clarity in segmentation examples
2024-11-19 16:49:25 +01:00
Marc Sun
ce1d328e3b
Fix cache_utils for optimum.quanto kvcache quantization ( #34750 )
...
* add co-author
Co-authored-by: w3rew <w3rew@users.noreply.github.com >
* fix docs
* fix cache
* remove print
---------
Co-authored-by: w3rew <w3rew@users.noreply.github.com >
2024-11-19 14:16:34 +01:00
Arthur
54739a320e
Self-speculation (Layer-Skip Llama) ( #34240 )
...
* 😅
* early exit (#34244 )
* mvp
* docs and tests
* a few fixes
* no shared cache
* Apply suggestions from code review
Co-authored-by: Mostafa Elhoushi <m.elhoushi@ieee.org >
* docs
* make fix-copies
* cohere fix
* [test all]
* [test all] consistent model code copies
* [test all] make fix-copies :D
* Apply suggestions from code review
Co-authored-by: Pedro Cuenca <pedro@huggingface.co >
Co-authored-by: Mostafa Elhoushi <m.elhoushi@ieee.org >
* Update src/transformers/generation/candidate_generator.py
* Update src/transformers/generation/configuration_utils.py
Co-authored-by: Pedro Cuenca <pedro@huggingface.co >
* [test all] don't use a stand-alone attribute; fix test
---------
Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com >
Co-authored-by: Joao Gante <joao@huggingface.co >
Co-authored-by: Mostafa Elhoushi <m.elhoushi@ieee.org >
Co-authored-by: Pedro Cuenca <pedro@huggingface.co >
2024-11-19 12:20:07 +00:00
Ke Wen
20142ab542
Simplify Tensor Parallel implementation with PyTorch TP ( #34184 )
...
* Simplify Tensor Parallel implementation with PyTorch TP
* Move tp_plan to config
* Lint
* Format and warning
* Disable copy-from check
* Conditionally get attr from config
* make fix-copies
* Move base_model_tp_plan to PretrainedConfig
* Move TP into from_pretrained
* Add device context for load
* Do not serialize
* Move _tp_plan setting to post_init
* Add has_tp_plan
* Add test_tp
* Add 'Multi-gpu inference' doc
* Add backward support for device type identification
* Auto-detect accelerator
* supports_tp_plan
* copyright year
* Fix copy
2024-11-18 19:51:49 +01:00
ecyht2
7df93d6ffb
fix: Wrong task mentioned in docs ( #34757 )
2024-11-18 18:42:28 +00:00
Hun-soo Jung
7693b62268
Fix callback key name ( #34762 )
...
Fixes typo.
2024-11-18 18:41:12 +00:00
Fanli Lin
e80a65ba4f
[tests] add XPU part to testing ( #34778 )
...
add XPU part to testing
Signed-off-by: Lin, Fanli <fanli.lin@intel.com >
2024-11-18 09:59:11 -08:00
Fanli Lin
9568a9dfc5
[docs] add XPU besides CUDA, MPS etc. ( #34777 )
...
add XPU
2024-11-18 09:58:50 -08:00
Fanli Lin
8568bf1bcf
[docs] make empty_cache device-agnostic ( #34774 )
...
make device-agnostic
2024-11-18 09:58:26 -08:00
Ofek Lev
eb0ab3ed4b
Fix broken link ( #34618 )
2024-11-18 14:13:26 +01:00
Raushan Turganbay
1646ffb4d1
VLMs: patch_size -> num_image_tokens in processing ( #33424 )
...
* use num additional tokens
* fix copies + docs
* another fix copies :)
* add docs
* move order for BC
2024-11-18 13:21:07 +01:00
Shane A
3ee24e2208
Add OLMo November 2024 ( #34551 )
...
* Add model skeletion with transformers-cli add-new-model-like
* Convert config to modular, add rms_norm_eps, delete clip_qkv
* Convert model to modular, add RMSNorm
* Add flash attention with qk norm and no qkv clipping
* Add decoder layer with RMSNorm after attention/feedforward layers
* Add base and causal model
* Add converter improvements from OLMo repo
* Update weight loading in OLMo to HF converter
* Set correct default for rms_norm_eps
* Set correct pipeline_model_mapping in test
* Run make fixup
* Fix model type
* Re-run modular conversion
* Manually set config docs to fix build errors
* Convert olmo-1124 to olmo_1124 to fix flash attention docs errors
* Start updating tests
* Update tests
* Copy upstream test_eager_matches_sdpa_inference_1_bfloat16 changes to olmo_1124
* Rename input_layernorm and post_attention_layernorm to reflect their ops better
* Use correct tokenizer
* Remove test unsupported by GPT2 tokenizer
* Create GenerationConfig outside of from_pretrained call
* Use simpler init file structure
* Add explicit __all__ to support simplified init
* Make safetensor serialization the default
* Update OLMo November 2024 docs
2024-11-18 10:43:10 +01:00
jiqing-feng
52ea4aa589
add xpu path for awq ( #34712 )
...
* add xpu path for awq
* update readme
2024-11-15 15:45:24 +01:00
Lysandre Debut
f5dbfab7f3
Update llava.md ( #34749 )
...
LLava -> Llava
2024-11-15 15:39:57 +01:00
Fanli Lin
a3d69a8994
[docs] add xpu device check ( #34684 )
...
* add XPU path
* use accelerate API
* Update docs/source/en/tasks/semantic_segmentation.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* update more places with accelerate API
---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
2024-11-13 14:16:59 -08:00
Pedro Cuenca
e7c36a9d57
[docs] Broken link in generation_strategies ( #34717 )
...
[docs] Broken link
2024-11-13 13:44:42 -08:00
MaCAT
be8748a53c
🌐 [i18n-KO] Translated marian.md to Korean ( #34698 )
...
* initial translation
* removed english
* Fixed Trivial Typos, updated _toctree.yml
2024-11-13 13:14:23 -08:00
Ahmed Almaghz
6de2a4d1f1
[i18n-ar] Translated file : docs/source/ar/torchscript.md into Arabic ( #33079 )
...
* Add docs/source/ar/torchscript.md to Add_docs_source_ar_torchscript.md
* Update docs/source/ar/torchscript.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/torchscript.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/torchscript.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/torchscript.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/torchscript.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/torchscript.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/torchscript.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/torchscript.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/torchscript.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/torchscript.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/torchscript.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/torchscript.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/torchscript.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/torchscript.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/torchscript.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/torchscript.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/torchscript.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/torchscript.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/torchscript.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/torchscript.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Merge troubleshooting.md with this Branch
* Update _toctree.yml
* Update torchscript.md
* Update troubleshooting.md
---------
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
2024-11-11 10:41:01 -08:00
Fanli Lin
25f510a9c6
[docs] update not-working model revision ( #34682 )
...
update revision
2024-11-11 07:09:31 -08:00
Aymeric Roucher
3ea3ab62d8
Agents: turn any Space into a Tool with Tool.from_space() ( #34561 )
...
* Agents: you can now load a Space as a tool
2024-11-10 12:22:40 +01:00
Ahmed Almaghz
768f3c016e
[i18n-ar] Translated file : docs/source/ar/trainer.md into Arabic ( #33080 )
...
* Add docs/source/ar/trainer.md to Add_docs_source_ar_trainer.md
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update trainer.md
* Update trainer.md
* Update trainer.md
* Create _toctree.yml
* Delete docs/source/ar/_toctree.yml
* Update _toctree.yml - add trainer
* Update _toctree.yml
* merge serialization.md into this branch
* merge sagemaker.md into this PR
* Update _toctree.yml
* Update docs/source/ar/trainer.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
---------
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
2024-11-09 11:26:28 -08:00
MaCAT
a06a0d1263
🌐 [i18n-KO] Translated bert.md to Korean ( #34627 )
...
* Translated bert.md, Need additional check
* Translation 2nd ver, changed _toctree.yml
* Fixed Typo
* Update bert.md
Co-authored-by: YONGSANG <71686691+4N3MONE@users.noreply.github.com >
* Update bert.md
Co-authored-by: YONGSANG <71686691+4N3MONE@users.noreply.github.com >
* Update bert.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update bert.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
---------
Co-authored-by: YONGSANG <71686691+4N3MONE@users.noreply.github.com >
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
2024-11-07 18:56:09 -08:00
Jiwook Han
1cf17077bf
🌐 [i18n-KO] Translated timesformer.md to Korean ( #33972 )
...
* docs: ko: model_doc/timesformer.md
* feat: nmt draft
* fix: manual edits
* fix_toctree
* fix toctree on Video Models
2024-11-07 11:04:27 -08:00
Ahnjj_DEV
7bbc624743
🌐 [i18n-KO] Translated convbert.md to Korean ( #34599 )
...
* docs: ko: convbert.md
* Update _toctree.yml
* feat: nmt draft
2024-11-05 09:32:17 -08:00
MaCAT
1112c54604
🌐 [i18n-KO] Translated perf_train_special.md to Korean ( #34590 )
...
* Translated to Ko, 1st version
* updated _toctree.yml
2024-11-04 09:41:44 -08:00
Karthik Vallamsetla
a86bd6f2d8
[i18n-HI] Translated TFLite page to Hindi ( #34572 )
...
* [i18n-HI] Translated TFLite page to Hindi
* [i18n-HI] Translated TFLite page to Hindi
* Update docs/source/hi/tflite.md
Co-authored-by: K.B.Dharun Krishna <kbdharunkrishna@gmail.com >
---------
Co-authored-by: K.B.Dharun Krishna <kbdharunkrishna@gmail.com >
2024-11-04 09:40:30 -08:00
Raushan Turganbay
187439c3fa
VLM: special multimodal Tokenizer ( #34461 )
...
* kinda works
* update
* add tests
* update
* use special tokens in processors
* typo
* fix copies
* fix
* fix moshi after rebase
* update
* fix tests
* update
* Update docs/source/en/main_classes/tokenizer.md
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com >
* update docs
* test for load time adding tokens
* fix some more tests which are now fetched better
* one more fix
---------
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com >
2024-11-04 16:37:51 +01:00
Karthik Vallamsetla
33868a057c
[i18n-HI] Translated accelerate page to Hindi ( #34443 )
...
* [i18n-HI] Translated accelerate page to Hindi
* Update docs/source/hi/accelerate.md
Co-authored-by: K.B.Dharun Krishna <kbdharunkrishna@gmail.com >
* Update docs/source/hi/accelerate.md
Co-authored-by: K.B.Dharun Krishna <kbdharunkrishna@gmail.com >
* Update docs/source/hi/accelerate.md
Co-authored-by: K.B.Dharun Krishna <kbdharunkrishna@gmail.com >
* Update docs/source/hi/accelerate.md
Co-authored-by: K.B.Dharun Krishna <kbdharunkrishna@gmail.com >
---------
Co-authored-by: Kay <kay@Kays-MacBook-Pro.local >
Co-authored-by: K.B.Dharun Krishna <kbdharunkrishna@gmail.com >
2024-11-01 08:26:45 -07:00
Ahmed Almaghz
b53e44e847
[i18n-ar] Translated file : docs/source/ar/multilingual.md into Arabic ( #33048 )
...
* Add docs/source/ar/multilingual.md to Add_docs_source_ar_multilingual.md
* Update docs/source/ar/multilingual.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/multilingual.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/multilingual.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/multilingual.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/multilingual.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/multilingual.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/multilingual.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/multilingual.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/multilingual.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/multilingual.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/multilingual.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/multilingual.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/multilingual.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/multilingual.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/multilingual.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/multilingual.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update _toctree.yml
* Update _toctree.yml
* Add Translated files to branch for merg
* Update _toctree.yml
* Update _toctree.yml
* Update custom_models.md
* Update chat_templating.md
* Update docs/source/ar/create_a_model.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update create_a_model.md
* Update gguf.md
* Update gguf.md
* Update gguf.md
* Update gguf.md
---------
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
2024-10-31 16:10:09 -07:00
jiqing-feng
2801d7bcf6
update doc ( #34478 )
...
* update doc
* Update docs/source/en/perf_train_cpu.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* delete closing tip
---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
2024-10-31 15:59:23 -07:00
Yoni Gozlan
203e27059b
Add image text to text pipeline ( #34170 )
...
* Standardize image-text-to-text-models-output
add post_process_image_text_to_text to chameleon and cleanup
Fix legacy kwarg behavior and deprecation warning
add post_process_image_text_to_text to qwen2_vl and llava_onevision
Add post_process_image_text_to_text to idefics3, mllama, pixtral processor
* nit var name post_process_image_text_to_text udop
* nit fix deprecation warnings
* Add image-text-to-text pipeline
* add support for image url in chat template for pipeline
* Reformat to be fully compatible with chat templates
* Add tests chat template
* Fix imports and tests
* Add pipeline tag
* change logic handling of single prompt ans multiple images
* add pipeline mapping to models
* fix batched inference
* fix tests
* Add manual batching for preprocessing
* Fix outputs with nested images
* Add support for all common processing kwargs
* Add default padding when multiple text inputs (batch size>1)
* nit change version deprecation warning
* Add support for text only inference
* add chat_template warnings
* Add pipeline tests and add copied from post process function
* Fix batched pipeline tests
* nit
* Fix pipeline tests blip2
* remove unnecessary max_new_tokens
* revert processing kosmos2 and remove unnecessary max_new_tokens
* fix pipeline tests idefics
* Force try loading processor if pipeline supports it
* revert load_processor change
* hardcode loading only processor
* remove unnecessary try except
* skip imagetexttotext tests for kosmos2 as tiny model causes problems
* Make code clearer
* Address review comments
* remove preprocessing logic from pipeline
* fix fuyu
* add BC resize fuyu
* Move post_process_image_text_to_text to ProcessorMixin
* add guard in post_process
* fix zero shot object detection pipeline
* add support for generator input in pipeline
* nit
* change default image-text-to-text model to llava onevision
* fix owlv2 size dict
* Change legacy deprecation warning to only show when True
2024-10-31 15:48:11 -04:00
Yoni Gozlan
48872fd6ae
Add Image Processor Fast RT-DETR ( #34354 )
...
* add fast image processor rtdetr
* add gpu/cpu test and fix docstring
* remove prints
* add to doc
* nit docstring
* avoid iterating over images/annotations several times
* change torch typing
* Add image processor fast documentation
2024-10-30 13:49:47 -04:00
Vladislav Bronzov
5251fe6271
Add GGUF for Mamba ( #34200 )
...
* add mamba architecture for gguf
* add logic for weights conversion, some fixes and refactoring
* add lm_head layers, unit test refactoring
* more fixes for tests
* remove lm_head creation
* remove unused comments
2024-10-30 16:52:17 +01:00
Raushan Turganbay
0f764a5af7
Mllama: update docs ( #34334 )
...
* update docs
* be more explicit
* use avaialble methods
2024-10-30 10:11:50 +01:00
Apoorv Khandelwal
e9ad460494
Adding optimizer_cls_and_kwargs to Trainer.__init__ ( #34358 )
...
* Adding `optimizer_cls_and_kwargs` to `Trainer.__init__`
* formatting
* make fix-copies docstring
* added more docs for optimizer_cls_and_kwargs
* add docs for Trainer(optimizer_cls_and_kwargs)
* reverting anchor names
2024-10-29 16:23:16 +01:00
Martin Gubri
626c610a4d
Fix perplexity computation in perplexity.md ( #34387 )
...
fix average NLL in perplexity.md
2024-10-29 11:10:10 +01:00
StevenBucaille
a1835195d1
🚨 🚨 🚨 [SuperPoint] Fix keypoint coordinate output and add post processing ( #33200 )
...
* feat: Added int conversion and unwrapping
* test: added tests for post_process_keypoint_detection of SuperPointImageProcessor
* docs: changed docs to include post_process_keypoint_detection method and switched from opencv to matplotlib
* test: changed test to not depend on SuperPointModel forward
* test: added missing require_torch decorator
* docs: changed pyplot parameters for the keypoints to be more visible in the example
* tests: changed import torch location to make test_flax and test_tf
* Revert "tests: changed import torch location to make test_flax and test_tf"
This reverts commit 39b32a2f69500bc7af01715fc7beae2260549afe.
* tests: fixed import
* chore: applied suggestions from code review
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com >
* tests: fixed import
* tests: fixed import (bis)
* tests: fixed import (ter)
* feat: added choice of type for target_size and changed tests accordingly
* docs: updated code snippet to reflect the addition of target size type choice in post process method
* tests: fixed imports (...)
* tests: fixed imports (...)
* style: formatting file
* docs: fixed typo from image[0] to image.size[0]
* docs: added output image and fixed some tests
* Update docs/source/en/model_doc/superpoint.md
Co-authored-by: Pavel Iakubovskii <qubvel@gmail.com >
* fix: included SuperPointKeypointDescriptionOutput in TYPE_CHECKING if statement and changed tests results to reflect changes to SuperPoint from absolute keypoints coordinates to relative
* docs: changed SuperPoint's docs to print output instead of just accessing
* style: applied make style
* docs: added missing output type and precision in docstring of post_process_keypoint_detection
* perf: deleted loop to perform keypoint conversion in one statement
* fix: moved keypoint conversion at the end of model forward
* docs: changed SuperPointInterestPointDecoder to SuperPointKeypointDecoder class name and added relative (x, y) coordinates information to its method
* fix: changed type hint
* refactor: removed unnecessary brackets
* revert: SuperPointKeypointDecoder to SuperPointInterestPointDecoder
* Update docs/source/en/model_doc/superpoint.md
Co-authored-by: Pavel Iakubovskii <qubvel@gmail.com >
---------
Co-authored-by: Steven Bucaille <steven.bucaille@buawei.com >
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com >
Co-authored-by: Pavel Iakubovskii <qubvel@gmail.com >
2024-10-29 09:36:03 +00:00
Ahmed Almaghz
a17f287ac0
[i18n-ar] Translated file : docs/source/ar/fast_tokenizers.md into Arabic ( #33034 )
...
* Add docs/source/ar/fast_tokenizers.md to Add_docs_source_ar_fast_tokenizers.md
* Update _toctree.yml
* Update _toctree.yml
* Update docs/source/ar/_toctree.yml
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/fast_tokenizers.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/fast_tokenizers.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/fast_tokenizers.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/fast_tokenizers.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/fast_tokenizers.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/fast_tokenizers.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/fast_tokenizers.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/fast_tokenizers.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/fast_tokenizers.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/fast_tokenizers.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
---------
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
2024-10-28 10:54:37 -07:00
wony617
1f7539c829
🌐 [i18n-KO] Translated model_doc/barthez.md to Korean ( #33980 )
...
* docs: ko: model_doc/barthez.md
* feat: nmt draft
---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
2024-10-28 10:46:49 -07:00
Rudy Delouya
6a62a6d1b5
Fix typos in agents_advanced.md ( #34405 )
2024-10-25 08:52:29 -07:00
Joao Gante
8814043c8c
SynthID: better example ( #34372 )
...
* better example
* Update src/transformers/generation/configuration_utils.py
* Update src/transformers/generation/logits_process.py
* nits
2024-10-25 11:46:46 +01:00
김준재
dd267fca72
Add T5 GGUF loading support ( #33389 )
...
* add: GGUFT5Converter
* add: tensormapping for t5
* add: test code for t5
* fix: Remove whitespace from blank line
* add: t5 fp16 tests
* fix: whitespace formatting
* fix: minor formatting
* fix: testing every weights
2024-10-24 15:10:59 +02:00
Thomas Furtner
30c76d5b28
add code generation to natural language processing section ( #34333 )
2024-10-24 14:42:47 +02:00
Lysandre Debut
2112027d0c
Zamba is an LM ( #34342 )
...
* Zamba is an LM
* Addition
2024-10-24 14:29:33 +02:00
blueingman
f0b3ef9e2e
translated gguf.md into chinese ( #34163 )
...
* translated gguf.md into chinese
* Apply suggestions from code review
I have updated the PR accordingly.Thank you very much for detailed guidance,and I 'll pay more attention to the details next time.
Co-authored-by: Isotr0py <2037008807@qq.com >
* Apply suggestions from code review
Co-authored-by: Isotr0py <2037008807@qq.com >
---------
Co-authored-by: Isotr0py <2037008807@qq.com >
2024-10-24 11:47:58 +02:00
Yih-Dar
f0e640adfa
Drop support for Python 3.8 ( #34314 )
...
* drop python 3.8
* update docker files
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com >
2024-10-24 11:16:55 +02:00
Abhishek Maurya
65753d6065
Remove graph breaks for torch.compile() in flash_attention_forward when Lllama Model is padding free tuned ( #33932 )
...
* fix: fixes for graph breaks
Signed-off-by: Abhishek <maurya.abhishek@ibm.com >
* fix: formatting
Signed-off-by: Abhishek <maurya.abhishek@ibm.com >
* fix: import error
Signed-off-by: Abhishek <maurya.abhishek@ibm.com >
* fix: Add Fa2Kwargs
Signed-off-by: Abhishek <maurya.abhishek@ibm.com >
* fix: PR Changes
Signed-off-by: Abhishek <maurya.abhishek@ibm.com >
* PR changes
Signed-off-by: Abhishek <maurya.abhishek@ibm.com >
* PR changes
Signed-off-by: Abhishek <maurya.abhishek@ibm.com >
* PR changes
Signed-off-by: Abhishek <maurya.abhishek@ibm.com >
* PR changes
Signed-off-by: Abhishek <maurya.abhishek@ibm.com >
* Revert "PR changes"
This reverts commit 39d2868e5c93cc5f3f3c7c6ff981b66614c0e0e4.
* PR changes
Signed-off-by: Abhishek <maurya.abhishek@ibm.com >
* fix: FlashAttentionKwarg
Signed-off-by: Abhishek <maurya.abhishek@ibm.com >
* fix: FlashAttentionKwarg
Signed-off-by: Abhishek <maurya.abhishek@ibm.com >
* PR Changes
Signed-off-by: Abhishek <maurya.abhishek@ibm.com >
* PR Changes
Signed-off-by: Abhishek <maurya.abhishek@ibm.com >
* PR Changes
Signed-off-by: Abhishek <maurya.abhishek@ibm.com >
* PR Changes
Signed-off-by: Abhishek <maurya.abhishek@ibm.com >
* PR Changes
Signed-off-by: Abhishek <maurya.abhishek@ibm.com >
* addition of documentation
Signed-off-by: Abhishek <maurya.abhishek@ibm.com >
* change in _flash_attention_forward
Signed-off-by: Abhishek <maurya.abhishek@ibm.com >
* make fix-copies
Signed-off-by: Abhishek <maurya.abhishek@ibm.com >
* revert make fix-copies
Signed-off-by: Abhishek <maurya.abhishek@ibm.com >
* fix copies
* style
* loss kwargs typing
* style and pull latest changes
---------
Signed-off-by: Abhishek <maurya.abhishek@ibm.com >
Co-authored-by: Arthur Zucker <arthur.zucker@gmail.com >
2024-10-24 11:02:54 +02:00
