blueingman
2910015d6d
[i18n-zh]Translated perf_train_special.md into Chinese ( #34948 )
...
* Add translation for perf_train_special documentation
* Update docs/source/zh/perf_train_special.md
Co-authored-by: Isotr0py <2037008807@qq.com >
* Update docs/source/zh/perf_train_special.md
Co-authored-by: Isotr0py <2037008807@qq.com >
* Update _toctree.yml
* Update _toctree.yml
* Update perf_train_special.md
* Update perf_train_special.md
---------
Co-authored-by: Isotr0py <2037008807@qq.com >
2024-11-27 07:57:43 -08:00
Fanli Lin
637225508f
[docs] add explanation to release_memory() ( #34911 )
...
* explain release_memory
* Update docs/source/en/llm_tutorial_optimization.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
2024-11-27 07:47:28 -08:00
MaCAT
0600f46353
🌐 [i18n-KO] Translated encoder-decoder.md to Korean ( #34880 )
...
* Initial version of translation, english still remaining
* Revised Translation, removed english. _toctree not updated
* updated _toctree.yml && 3rd ver translation
* updated _toctree.yml && 3rd ver translation
* Update encoder-decoder.md
Co-authored-by: YONGSANG <71686691+4N3MONE@users.noreply.github.com >
* Update encoder-decoder.md
Co-authored-by: YONGSANG <71686691+4N3MONE@users.noreply.github.com >
* Update encoder-decoder.md
Co-authored-by: YONGSANG <71686691+4N3MONE@users.noreply.github.com >
* Update encoder-decoder.md
Co-authored-by: YONGSANG <71686691+4N3MONE@users.noreply.github.com >
* Update encoder-decoder.md
Co-authored-by: YONGSANG <71686691+4N3MONE@users.noreply.github.com >
* Update encoder-decoder.md
Co-authored-by: YONGSANG <71686691+4N3MONE@users.noreply.github.com >
---------
Co-authored-by: YONGSANG <71686691+4N3MONE@users.noreply.github.com >
2024-11-27 07:47:14 -08:00
blueingman
6c3f168b36
[i18n-zh]Translated tiktoken.md into chinese ( #34936 )
...
* Add translation for tiktoken documentation
* Update tiktoken.md
* Update tiktoken.md
2024-11-26 10:09:52 -08:00
谭九鼎
5bfb40bc8e
docs: HUGGINGFACE_HUB_CACHE -> HF_HUB_CACHE ( #34904 )
2024-11-26 09:37:18 -08:00
Fanli Lin
784d22078a
[doc] use full path for run_qa.py ( #34914 )
...
use full path for run_qa.py
2024-11-26 09:23:44 -08:00
Fanli Lin
6bc0c219c1
[docs] use device-agnostic API instead of cuda ( #34913 )
...
add device-agnostic API
Signed-off-by: Lin, Fanli <fanli.lin@intel.com >
2024-11-26 09:23:34 -08:00
Ahmed Almaghz
64b73e61f8
[i18n-ar] Translated file : docs/source/ar/benchmarks.md into Arabic ( #33023 )
...
* Add docs/source/ar/benchmarks.md to Add_docs_source_ar_benchmarks.md
* Update docs/source/ar/benchmarks.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/benchmarks.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/benchmarks.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/benchmarks.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/benchmarks.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/benchmarks.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/benchmarks.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/benchmarks.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/benchmarks.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/benchmarks.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/benchmarks.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update _toctree.yml
* Update benchmarks.md
---------
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
2024-11-26 09:23:11 -08:00
Viktor Scherbakov
95c10fedb3
Updated documentation and added conversion utility ( #34319 )
...
* Updated documentation and added conversion utility
* Update docs/source/en/tiktoken.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update docs/source/en/tiktoken.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Moved util function to integration folder + allow for str
* Update formatting
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com >
* Updated formatting
* style changes
---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com >
2024-11-25 18:44:09 +01:00
Shane A
9121ab8fe8
Rename OLMo November to OLMo2 ( #34864 )
...
* Rename/move OLMo Nov files to OLMo2
* Rename Olmo1124 and its variants to Olmo2
2024-11-25 16:31:22 +01:00
farrosalferro
c57eafdaa1
Add Nemotron GGUF Loading Support ( #34725 )
...
* Add Nemotron GGUF Loading Support
* fix the Nemotron architecture assignation
---------
Co-authored-by: Marc Sun <57196510+SunMarc@users.noreply.github.com >
2024-11-21 11:37:34 +01:00
wwwbai
3033509327
Translate attention.md into Chinese ( #34716 )
...
* try
* tryagain
* tryagggain
* translated
* translated2
* Update docs/source/zh/attention.md
Co-authored-by: Huazhong Ji <hzji210@gmail.com >
---------
Co-authored-by: Huazhong Ji <hzji210@gmail.com >
2024-11-19 10:03:12 -08:00
Merve Noyan
befbbf2f98
Added image-text-to-text pipeline to task guide ( #34783 )
...
* Added image-text-to-text pipeline to task guide
* Update docs/source/en/tasks/image_text_to_text.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update docs/source/en/tasks/image_text_to_text.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update docs/source/en/tasks/image_text_to_text.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update docs/source/en/tasks/image_text_to_text.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Merge codeblocks
---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
2024-11-19 09:49:10 -08:00
Yoni Gozlan
eedc113914
Add Image Processor Fast Deformable DETR ( #34353 )
...
* add deformable detr image processor fast
* add fast processor to doc
* fix copies
* nit docstring
* Add tests gpu/cpu and fix docstrings
* fix docstring
* import changes from detr
* fix imports
* rebase and fix
* fix input data format change in detr and rtdetr fast
2024-11-19 11:18:58 -05:00
David Zhang
427b62ed1a
Fix post process function called in the instance segmentation example of mask2former ( #34588 )
...
* Fix post process function called in the instance segmentation example of mask2former
* fix description and additional notes for post_process_instance_segmentation of maskformers
* remove white space in maskformers post_process_instance_segmentation doc
* change image.size[::-1] to height and width for clarity in segmentation examples
2024-11-19 16:49:25 +01:00
Marc Sun
ce1d328e3b
Fix cache_utils for optimum.quanto kvcache quantization ( #34750 )
...
* add co-author
Co-authored-by: w3rew <w3rew@users.noreply.github.com >
* fix docs
* fix cache
* remove print
---------
Co-authored-by: w3rew <w3rew@users.noreply.github.com >
2024-11-19 14:16:34 +01:00
Arthur
54739a320e
Self-speculation (Layer-Skip Llama) ( #34240 )
...
* 😅
* early exit (#34244 )
* mvp
* docs and tests
* a few fixes
* no shared cache
* Apply suggestions from code review
Co-authored-by: Mostafa Elhoushi <m.elhoushi@ieee.org >
* docs
* make fix-copies
* cohere fix
* [test all]
* [test all] consistent model code copies
* [test all] make fix-copies :D
* Apply suggestions from code review
Co-authored-by: Pedro Cuenca <pedro@huggingface.co >
Co-authored-by: Mostafa Elhoushi <m.elhoushi@ieee.org >
* Update src/transformers/generation/candidate_generator.py
* Update src/transformers/generation/configuration_utils.py
Co-authored-by: Pedro Cuenca <pedro@huggingface.co >
* [test all] don't use a stand-alone attribute; fix test
---------
Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com >
Co-authored-by: Joao Gante <joao@huggingface.co >
Co-authored-by: Mostafa Elhoushi <m.elhoushi@ieee.org >
Co-authored-by: Pedro Cuenca <pedro@huggingface.co >
2024-11-19 12:20:07 +00:00
Ke Wen
20142ab542
Simplify Tensor Parallel implementation with PyTorch TP ( #34184 )
...
* Simplify Tensor Parallel implementation with PyTorch TP
* Move tp_plan to config
* Lint
* Format and warning
* Disable copy-from check
* Conditionally get attr from config
* make fix-copies
* Move base_model_tp_plan to PretrainedConfig
* Move TP into from_pretrained
* Add device context for load
* Do not serialize
* Move _tp_plan setting to post_init
* Add has_tp_plan
* Add test_tp
* Add 'Multi-gpu inference' doc
* Add backward support for device type identification
* Auto-detect accelerator
* supports_tp_plan
* copyright year
* Fix copy
2024-11-18 19:51:49 +01:00
ecyht2
7df93d6ffb
fix: Wrong task mentioned in docs ( #34757 )
2024-11-18 18:42:28 +00:00
Hun-soo Jung
7693b62268
Fix callback key name ( #34762 )
...
Fixes typo.
2024-11-18 18:41:12 +00:00
Fanli Lin
e80a65ba4f
[tests] add XPU part to testing ( #34778 )
...
add XPU part to testing
Signed-off-by: Lin, Fanli <fanli.lin@intel.com >
2024-11-18 09:59:11 -08:00
Fanli Lin
9568a9dfc5
[docs] add XPU besides CUDA, MPS etc. ( #34777 )
...
add XPU
2024-11-18 09:58:50 -08:00
Fanli Lin
8568bf1bcf
[docs] make empty_cache device-agnostic ( #34774 )
...
make device-agnostic
2024-11-18 09:58:26 -08:00
Ofek Lev
eb0ab3ed4b
Fix broken link ( #34618 )
2024-11-18 14:13:26 +01:00
Raushan Turganbay
1646ffb4d1
VLMs: patch_size -> num_image_tokens in processing ( #33424 )
...
* use num additional tokens
* fix copies + docs
* another fix copies :)
* add docs
* move order for BC
2024-11-18 13:21:07 +01:00
Shane A
3ee24e2208
Add OLMo November 2024 ( #34551 )
...
* Add model skeletion with transformers-cli add-new-model-like
* Convert config to modular, add rms_norm_eps, delete clip_qkv
* Convert model to modular, add RMSNorm
* Add flash attention with qk norm and no qkv clipping
* Add decoder layer with RMSNorm after attention/feedforward layers
* Add base and causal model
* Add converter improvements from OLMo repo
* Update weight loading in OLMo to HF converter
* Set correct default for rms_norm_eps
* Set correct pipeline_model_mapping in test
* Run make fixup
* Fix model type
* Re-run modular conversion
* Manually set config docs to fix build errors
* Convert olmo-1124 to olmo_1124 to fix flash attention docs errors
* Start updating tests
* Update tests
* Copy upstream test_eager_matches_sdpa_inference_1_bfloat16 changes to olmo_1124
* Rename input_layernorm and post_attention_layernorm to reflect their ops better
* Use correct tokenizer
* Remove test unsupported by GPT2 tokenizer
* Create GenerationConfig outside of from_pretrained call
* Use simpler init file structure
* Add explicit __all__ to support simplified init
* Make safetensor serialization the default
* Update OLMo November 2024 docs
2024-11-18 10:43:10 +01:00
jiqing-feng
52ea4aa589
add xpu path for awq ( #34712 )
...
* add xpu path for awq
* update readme
2024-11-15 15:45:24 +01:00
Lysandre Debut
f5dbfab7f3
Update llava.md ( #34749 )
...
LLava -> Llava
2024-11-15 15:39:57 +01:00
Fanli Lin
a3d69a8994
[docs] add xpu device check ( #34684 )
...
* add XPU path
* use accelerate API
* Update docs/source/en/tasks/semantic_segmentation.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* update more places with accelerate API
---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
2024-11-13 14:16:59 -08:00
Pedro Cuenca
e7c36a9d57
[docs] Broken link in generation_strategies ( #34717 )
...
[docs] Broken link
2024-11-13 13:44:42 -08:00
MaCAT
be8748a53c
🌐 [i18n-KO] Translated marian.md to Korean ( #34698 )
...
* initial translation
* removed english
* Fixed Trivial Typos, updated _toctree.yml
2024-11-13 13:14:23 -08:00
Ahmed Almaghz
6de2a4d1f1
[i18n-ar] Translated file : docs/source/ar/torchscript.md into Arabic ( #33079 )
...
* Add docs/source/ar/torchscript.md to Add_docs_source_ar_torchscript.md
* Update docs/source/ar/torchscript.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/torchscript.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/torchscript.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/torchscript.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/torchscript.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/torchscript.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/torchscript.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/torchscript.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/torchscript.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/torchscript.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/torchscript.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/torchscript.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/torchscript.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/torchscript.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/torchscript.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/torchscript.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/torchscript.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/torchscript.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/torchscript.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/torchscript.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Merge troubleshooting.md with this Branch
* Update _toctree.yml
* Update torchscript.md
* Update troubleshooting.md
---------
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
2024-11-11 10:41:01 -08:00
Fanli Lin
25f510a9c6
[docs] update not-working model revision ( #34682 )
...
update revision
2024-11-11 07:09:31 -08:00
Aymeric Roucher
3ea3ab62d8
Agents: turn any Space into a Tool with Tool.from_space() ( #34561 )
...
* Agents: you can now load a Space as a tool
2024-11-10 12:22:40 +01:00
Ahmed Almaghz
768f3c016e
[i18n-ar] Translated file : docs/source/ar/trainer.md into Arabic ( #33080 )
...
* Add docs/source/ar/trainer.md to Add_docs_source_ar_trainer.md
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update trainer.md
* Update trainer.md
* Update trainer.md
* Create _toctree.yml
* Delete docs/source/ar/_toctree.yml
* Update _toctree.yml - add trainer
* Update _toctree.yml
* merge serialization.md into this branch
* merge sagemaker.md into this PR
* Update _toctree.yml
* Update docs/source/ar/trainer.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update docs/source/ar/trainer.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
---------
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
2024-11-09 11:26:28 -08:00
MaCAT
a06a0d1263
🌐 [i18n-KO] Translated bert.md to Korean ( #34627 )
...
* Translated bert.md, Need additional check
* Translation 2nd ver, changed _toctree.yml
* Fixed Typo
* Update bert.md
Co-authored-by: YONGSANG <71686691+4N3MONE@users.noreply.github.com >
* Update bert.md
Co-authored-by: YONGSANG <71686691+4N3MONE@users.noreply.github.com >
* Update bert.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update bert.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
---------
Co-authored-by: YONGSANG <71686691+4N3MONE@users.noreply.github.com >
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
2024-11-07 18:56:09 -08:00
Jiwook Han
1cf17077bf
🌐 [i18n-KO] Translated timesformer.md to Korean ( #33972 )
...
* docs: ko: model_doc/timesformer.md
* feat: nmt draft
* fix: manual edits
* fix_toctree
* fix toctree on Video Models
2024-11-07 11:04:27 -08:00
Ahnjj_DEV
7bbc624743
🌐 [i18n-KO] Translated convbert.md to Korean ( #34599 )
...
* docs: ko: convbert.md
* Update _toctree.yml
* feat: nmt draft
2024-11-05 09:32:17 -08:00
MaCAT
1112c54604
🌐 [i18n-KO] Translated perf_train_special.md to Korean ( #34590 )
...
* Translated to Ko, 1st version
* updated _toctree.yml
2024-11-04 09:41:44 -08:00
Karthik Vallamsetla
a86bd6f2d8
[i18n-HI] Translated TFLite page to Hindi ( #34572 )
...
* [i18n-HI] Translated TFLite page to Hindi
* [i18n-HI] Translated TFLite page to Hindi
* Update docs/source/hi/tflite.md
Co-authored-by: K.B.Dharun Krishna <kbdharunkrishna@gmail.com >
---------
Co-authored-by: K.B.Dharun Krishna <kbdharunkrishna@gmail.com >
2024-11-04 09:40:30 -08:00
Raushan Turganbay
187439c3fa
VLM: special multimodal Tokenizer ( #34461 )
...
* kinda works
* update
* add tests
* update
* use special tokens in processors
* typo
* fix copies
* fix
* fix moshi after rebase
* update
* fix tests
* update
* Update docs/source/en/main_classes/tokenizer.md
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com >
* update docs
* test for load time adding tokens
* fix some more tests which are now fetched better
* one more fix
---------
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com >
2024-11-04 16:37:51 +01:00
Karthik Vallamsetla
33868a057c
[i18n-HI] Translated accelerate page to Hindi ( #34443 )
...
* [i18n-HI] Translated accelerate page to Hindi
* Update docs/source/hi/accelerate.md
Co-authored-by: K.B.Dharun Krishna <kbdharunkrishna@gmail.com >
* Update docs/source/hi/accelerate.md
Co-authored-by: K.B.Dharun Krishna <kbdharunkrishna@gmail.com >
* Update docs/source/hi/accelerate.md
Co-authored-by: K.B.Dharun Krishna <kbdharunkrishna@gmail.com >
* Update docs/source/hi/accelerate.md
Co-authored-by: K.B.Dharun Krishna <kbdharunkrishna@gmail.com >
---------
Co-authored-by: Kay <kay@Kays-MacBook-Pro.local >
Co-authored-by: K.B.Dharun Krishna <kbdharunkrishna@gmail.com >
2024-11-01 08:26:45 -07:00
Ahmed Almaghz
b53e44e847
[i18n-ar] Translated file : docs/source/ar/multilingual.md into Arabic ( #33048 )
...
* Add docs/source/ar/multilingual.md to Add_docs_source_ar_multilingual.md
* Update docs/source/ar/multilingual.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/multilingual.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/multilingual.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/multilingual.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/multilingual.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/multilingual.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/multilingual.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/multilingual.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/multilingual.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/multilingual.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/multilingual.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/multilingual.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/multilingual.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/multilingual.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/multilingual.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/multilingual.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update _toctree.yml
* Update _toctree.yml
* Add Translated files to branch for merg
* Update _toctree.yml
* Update _toctree.yml
* Update custom_models.md
* Update chat_templating.md
* Update docs/source/ar/create_a_model.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update create_a_model.md
* Update gguf.md
* Update gguf.md
* Update gguf.md
* Update gguf.md
---------
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
2024-10-31 16:10:09 -07:00
jiqing-feng
2801d7bcf6
update doc ( #34478 )
...
* update doc
* Update docs/source/en/perf_train_cpu.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* delete closing tip
---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
2024-10-31 15:59:23 -07:00
Yoni Gozlan
203e27059b
Add image text to text pipeline ( #34170 )
...
* Standardize image-text-to-text-models-output
add post_process_image_text_to_text to chameleon and cleanup
Fix legacy kwarg behavior and deprecation warning
add post_process_image_text_to_text to qwen2_vl and llava_onevision
Add post_process_image_text_to_text to idefics3, mllama, pixtral processor
* nit var name post_process_image_text_to_text udop
* nit fix deprecation warnings
* Add image-text-to-text pipeline
* add support for image url in chat template for pipeline
* Reformat to be fully compatible with chat templates
* Add tests chat template
* Fix imports and tests
* Add pipeline tag
* change logic handling of single prompt ans multiple images
* add pipeline mapping to models
* fix batched inference
* fix tests
* Add manual batching for preprocessing
* Fix outputs with nested images
* Add support for all common processing kwargs
* Add default padding when multiple text inputs (batch size>1)
* nit change version deprecation warning
* Add support for text only inference
* add chat_template warnings
* Add pipeline tests and add copied from post process function
* Fix batched pipeline tests
* nit
* Fix pipeline tests blip2
* remove unnecessary max_new_tokens
* revert processing kosmos2 and remove unnecessary max_new_tokens
* fix pipeline tests idefics
* Force try loading processor if pipeline supports it
* revert load_processor change
* hardcode loading only processor
* remove unnecessary try except
* skip imagetexttotext tests for kosmos2 as tiny model causes problems
* Make code clearer
* Address review comments
* remove preprocessing logic from pipeline
* fix fuyu
* add BC resize fuyu
* Move post_process_image_text_to_text to ProcessorMixin
* add guard in post_process
* fix zero shot object detection pipeline
* add support for generator input in pipeline
* nit
* change default image-text-to-text model to llava onevision
* fix owlv2 size dict
* Change legacy deprecation warning to only show when True
2024-10-31 15:48:11 -04:00
Yoni Gozlan
48872fd6ae
Add Image Processor Fast RT-DETR ( #34354 )
...
* add fast image processor rtdetr
* add gpu/cpu test and fix docstring
* remove prints
* add to doc
* nit docstring
* avoid iterating over images/annotations several times
* change torch typing
* Add image processor fast documentation
2024-10-30 13:49:47 -04:00
Vladislav Bronzov
5251fe6271
Add GGUF for Mamba ( #34200 )
...
* add mamba architecture for gguf
* add logic for weights conversion, some fixes and refactoring
* add lm_head layers, unit test refactoring
* more fixes for tests
* remove lm_head creation
* remove unused comments
2024-10-30 16:52:17 +01:00
Raushan Turganbay
0f764a5af7
Mllama: update docs ( #34334 )
...
* update docs
* be more explicit
* use avaialble methods
2024-10-30 10:11:50 +01:00
Apoorv Khandelwal
e9ad460494
Adding optimizer_cls_and_kwargs to Trainer.__init__ ( #34358 )
...
* Adding `optimizer_cls_and_kwargs` to `Trainer.__init__`
* formatting
* make fix-copies docstring
* added more docs for optimizer_cls_and_kwargs
* add docs for Trainer(optimizer_cls_and_kwargs)
* reverting anchor names
2024-10-29 16:23:16 +01:00
Martin Gubri
626c610a4d
Fix perplexity computation in perplexity.md ( #34387 )
...
fix average NLL in perplexity.md
2024-10-29 11:10:10 +01:00
