Yoni Gozlan
5f0c181f4e
Uniformize kwargs for image-text-to-text processors ( #32544 )
...
* uniformize FUYU processor kwargs
* Uniformize instructblip processor kwargs
* Fix processor kwargs and tests Fuyu, InstructBlip, Kosmos2
* Uniformize llava_next processor
* Fix save_load test for processor with chat_template only as extra init args
* Fix import Unpack
* Fix Fuyu Processor import
* Fix FuyuProcessor import
* Fix FuyuProcessor
* Add defaults for specific kwargs kosmos2
* Fix Udop to return BatchFeature instead of BatchEncoding and uniformize kwargs
* Add tests processor Udop
* remove Copied from in processing Udop as change of input orders caused by BatchEncoding -> BatchFeature
* Fix overwrite tests kwargs processors
* Add warnings and BC for changes in processor inputs order, change docs, add BC for text_pair as arg for Udop
* Fix processing test fuyu
* remove unnecessary pad_token check in instructblip ProcessorTest
* Fix BC tests and cleanup
* FIx imports fuyu
* Uniformize Pix2Struct
* Fix wrong name for FuyuProcessorKwargs
* Fix slow tests reversed inputs align fuyu llava-next, change udop warning
* Fix wrong logging import udop
* Add check images text input order
* Fix copies
* change text pair handling when positional arg
* rebase on main, fix imports in test_processing_common
* remove optional args and udop uniformization from this PR
* fix failing tests
* remove unnecessary test, fix processing utils and test processing common
* cleanup Unpack
* cleanup
* fix conflict grounding dino
2024-09-24 21:28:19 -04:00
Arthur
317e069ee7
Modular transformers: modularity and inheritance for new model additions ( #33248 )
...
* update exampel
* update
* push the converted diff files for testing and ci
* correct one example
* fix class attributes and docstring
* nits
* oups
* fixed config!
* update
* nitd
* class attributes are not matched against the other, this is missing
* fixed overwriting self.xxx now onto the attributes I think
* partial fix, now order with docstring
* fix docstring order?
* more fixes
* update
* fix missing docstrings!
* examples don't all work yet
* fixup
* nit
* updated
* hick
* update
* delete
* update
* update
* update
* fix
* all default
* no local import
* fix more diff
* some fix related to "safe imports"
* push fixed
* add helper!
* style
* add a check
* all by default
* add the
* update
* FINALLY!
* nit
* fix config dependencies
* man that is it
* fix fix
* update diffs
* fix the last issue
* re-default to all
* alll the fixes
* nice
* fix properties vs setter
* fixup
* updates
* update dependencies
* make sure to install what needs to be installed
* fixup
* quick fix for now
* fix!
* fixup
* update
* update
* updates
* whitespaces
* nit
* fix
* simplify everything, and make it file agnostic (should work for image processors)
* style
* finish fixing all import issues
* fixup
* empty modeling should not be written!
* Add logic to find who depends on what
* update
* cleanup
* update
* update gemma to support positions
* some small nits
* this is the correct docstring for gemma2
* fix merging of docstrings
* update
* fixup
* update
* take doc into account
* styling
* update
* fix hidden activation
* more fixes
* final fixes!
* fixup
* fixup instruct blip video
* update
* fix bugs
* align gemma2 with the rest as well
* updats
* revert
* update
* more reversiom
* grind
* more
* arf
* update
* order will matter
* finish del stuff
* update
* rename to modular
* fixup
* nits
* update makefile
* fixup
* update order of the checks!
* fix
* fix docstring that has a call inside
* fiix conversion check
* style
* add some initial documentation
* update
* update doc
* some fixup
* updates
* yups
* Mostly todo gimme a minut
* update
* fixup
* revert some stuff
* Review docs for the modular transformers (#33472 )
Docs
* good update
* fixup
* mmm current updates lead to this code
* okay, this fixes it
* cool
* fixes
* update
* nit
* updates
* nits
* fix doc
* update
* revert bad changes
* update
* updates
* proper update
* update
* update?
* up
* update
* cool
* nits
* nits
* bon bon
* fix
* ?
* minimise changes
* update
* update
* update
* updates?
* fixed gemma2
* kind of a hack
* nits
* update
* remove `diffs` in favor of `modular`
* fix make fix copies
---------
Co-authored-by: Lysandre Debut <hi@lysand.re >
2024-09-24 15:54:07 +02:00
jiqing-feng
11c27dd331
Enable BNB multi-backend support ( #31098 )
...
* enable cpu bnb path
* fix style
* fix code style
* fix 4 bit path
* Update src/transformers/utils/import_utils.py
Co-authored-by: Aarni Koskela <akx@iki.fi >
* add multi backend refactor tests
* fix style
* tweak 4bit quantizer + fix corresponding tests
* tweak 8bit quantizer + *try* fixing corresponding tests
* fix dequant bnb 8bit
* account for Intel CPU in variability of expected outputs
* enable cpu and xpu device map
* further tweaks to account for Intel CPU
* fix autocast to work with both cpu + cuda
* fix comments
* fix comments
* switch to testing_utils.torch_device
* allow for xpu in multi-gpu tests
* fix tests 4bit for CPU NF4
* fix bug with is_torch_xpu_available needing to be called as func
* avoid issue where test reports attr err due to other failure
* fix formatting
* fix typo from resolving of merge conflict
* polish based on last PR review
Co-authored-by: Marc Sun <57196510+SunMarc@users.noreply.github.com >
* fix CI
* Update src/transformers/integrations/integration_utils.py
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com >
* Update src/transformers/integrations/integration_utils.py
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com >
* fix error log
* fix error msg
* add \n in error log
* make quality
* rm bnb cuda restriction in doc
* cpu model don't need dispatch
* fix doc
* fix style
* check cuda avaliable in testing
* fix tests
* Update docs/source/en/model_doc/chameleon.md
Co-authored-by: Marc Sun <57196510+SunMarc@users.noreply.github.com >
* Update docs/source/en/model_doc/llava_next.md
Co-authored-by: Aarni Koskela <akx@iki.fi >
* Update tests/quantization/bnb/test_4bit.py
Co-authored-by: Aarni Koskela <akx@iki.fi >
* Update tests/quantization/bnb/test_4bit.py
Co-authored-by: Aarni Koskela <akx@iki.fi >
* fix doc
* fix check multibackends
* fix import sort
* remove check torch in bnb
* docs: update bitsandbytes references with multi-backend info
* docs: fix small mistakes in bnb paragraph
* run formatting
* reveret bnb check
* move bnb multi-backend check to import_utils
* Update src/transformers/utils/import_utils.py
Co-authored-by: Aarni Koskela <akx@iki.fi >
* fix bnb check
* minor fix for bnb
* check lib first
* fix code style
* Revert "run formatting"
This reverts commit ac108c6d6b34f45a5745a736ba57282405cfaa61.
* fix format
* give warning when bnb version is low and no cuda found]
* fix device assignment check to be multi-device capable
* address akx feedback on get_avlbl_dev fn
* revert partially, as we don't want the function that public, as docs would be too much (enforced)
---------
Co-authored-by: Aarni Koskela <akx@iki.fi >
Co-authored-by: Titus von Koeller <9048635+Titus-von-Koeller@users.noreply.github.com >
Co-authored-by: Marc Sun <57196510+SunMarc@users.noreply.github.com >
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com >
2024-09-24 03:40:56 -06:00
Avishai Elmakies
78b2929c05
Sdpa dino v2 ( #33403 )
...
* add sdpa to dinov2
* fixup
* add dinov2 to sdpa doc
* update doc order
* [run-slow] dinov2
* common to eager
* [run-slow] dinov2
* update attn implementation in common
* update test_modeling_dinov2 to have mask_ration, num_masks and mask_length similar to vit
* [run-slow] dinov2
---------
Co-authored-by: Avishai Elmakies <avishai.elma@cs.huji.ac.il >
2024-09-21 01:58:00 +01:00
amyeroberts
e71bf70e33
Pixtral update example checkpoint ( #33633 )
...
* Update pixtral example checkpoint
* Fix typo
2024-09-21 01:01:16 +01:00
Mayank Mishra
e472e077c2
Granitemoe ( #33207 )
...
* first commit
* drop tokenizer
* drop tokenizer
* drop tokenizer
* drop convert
* granite
* drop tokenization test
* mup
* fix
* reformat
* reformat
* reformat
* fix docs
* stop checking for checkpoint
* update support
* attention multiplier
* update model
* tiny drop
* saibo drop
* skip test
* fix test
* fix test
* drop
* drop useless imports
* update docs
* drop flash function
* copied from
* drop pretraining tp
* drop pretraining tp
* drop pretraining tp
* drop unused import
* drop code path
* change name
* softmax scale
* head dim
* drop legacy cache
* rename params
* cleanup
* fix copies
* comments
* add back legacy cache
* multipliers
* multipliers
* multipliers
* text fix
* fix copies
* merge
* multipliers
* attention multiplier
* drop unused imports
* add granitemoe
* add decoration
* remove moe from sequenceclassification
* fix test
* fix
* fix
* fix
* move rope?
* merge
* drop bias
* drop bias
* Update src/transformers/models/granite/configuration_granite.py
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com >
* fix
* Update src/transformers/models/granite/modeling_granite.py
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com >
* fix
* fix
* fix
* fix
* drop
* drop
* fix
* fix
* cleanup
* cleanup
* fix
* fix granite tests
* fp32 test
* fix
* drop jitter
* fix
* rename
* rename
* fix config
* add gen test
---------
Co-authored-by: Yikang Shen <yikang.shn@gmail.com >
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com >
2024-09-21 01:43:50 +02:00
Omar Salman
653eb40425
Add sdpa for BioGpt ( #33592 )
...
* Add sdpa for BioGpt
* Updates
* Add the docs
* [run_slow] biogpt
* Use the copy mechanism to ensure consistency
* [run_slow] biogpt
2024-09-20 14:27:32 +01:00
Yoni Gozlan
f111d5b783
Uniformize kwargs for Paligemma processor and update docs ( #33571 )
...
* Uniformize paligemma processor
* nit
2024-09-19 14:14:06 -04:00
Joao Gante
80b774eb29
Cache: don't show warning in forward passes when past_key_values is None ( #33541 )
2024-09-19 12:02:46 +01:00
Yoach Lacombe
5af7d41e49
Codec integration ( #33565 )
...
* clean mimi commit
* some nits suggestions from Arthur
* make fixup
* rename repo id + change readme
* Update docs/source/en/model_doc/mimi.md
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com >
* add flaky flag to batching equivalence due to audio_codes failing sometimes
---------
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com >
2024-09-18 19:23:44 +02:00
Aymeric Roucher
e6d9f39dd7
Decorator for easier tool building ( #33439 )
...
* Decorator for tool building
2024-09-18 11:07:51 +02:00
Yoni Gozlan
d8500cd229
Uniformize kwargs for Pixtral processor ( #33521 )
...
* add uniformized pixtral and kwargs
* update doc
* fix _validate_images_text_input_order
* nit
2024-09-17 14:44:27 -04:00
Antoine Dussolle
763548427d
Add explicit example for RAG chat templating ( #33503 )
...
* Add explicit example for RAG chat templating
* Add Tip box and reformulate
Co-authored-by: Matt <Rocketknight1@users.noreply.github.com >
---------
Co-authored-by: Matt <Rocketknight1@users.noreply.github.com >
2024-09-17 16:08:05 +01:00
Max Buckley
ac5a0556f1
Update chameleon.md — fix runtime type error ( #33494 )
...
Update chameleon.md
Fix error
RuntimeError: Input type (float) and bias type (c10::BFloat16) should be the same
2024-09-17 13:32:49 +02:00
Ahmed Almaghz
c2d05897bf
[i18n-ar] Add File : docs/source/ar/_toctree.yml ( #32696 )
...
* Update ar lang build_documentation.yml
* Update ar lang build_pr_documentation.yml
* Update docs/source/ar/pipeline_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/pipeline_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/pipeline_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/pipeline_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/pipeline_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/pipeline_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/pipeline_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/pipeline_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/pipeline_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/pipeline_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/pipeline_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/pipeline_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/pipeline_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/pipeline_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/pipeline_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/pipeline_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/pipeline_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/pipeline_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/pipeline_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/pipeline_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/pipeline_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/pipeline_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/pipeline_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/pipeline_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/pipeline_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/autoclass_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/autoclass_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/autoclass_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/autoclass_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/autoclass_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/autoclass_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/autoclass_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/autoclass_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/autoclass_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/autoclass_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/autoclass_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/autoclass_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/autoclass_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/autoclass_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/preprocessing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/training.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/training.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/training.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/training.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/training.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/training.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/training.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/training.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/training.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/training.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/training.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/training.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/training.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/training.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/training.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/training.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/training.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/training.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/training.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/training.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/run_scripts.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/run_scripts.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/run_scripts.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/run_scripts.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/run_scripts.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/run_scripts.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/run_scripts.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/accelerate.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/accelerate.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/accelerate.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/accelerate.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/accelerate.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/accelerate.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Create _config.py
* Update _toctree.yml
* Update _toctree.yml
* Update docs/source/ar/peft.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/peft.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/peft.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/peft.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/peft.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/peft.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/peft.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/peft.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/peft.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update _toctree.yml
* Update docs/source/ar/model_sharing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/model_sharing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/model_sharing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/model_sharing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/model_sharing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/model_sharing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/model_sharing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/model_sharing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/model_sharing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/model_sharing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/model_sharing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/model_sharing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/model_sharing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/model_sharing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/model_sharing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/model_sharing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/model_sharing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/model_sharing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/model_sharing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/model_sharing.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/conversations.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/conversations.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/conversations.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/conversations.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/conversations.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/conversations.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/conversations.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/conversations.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/conversations.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/conversations.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/conversations.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/conversations.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/conversations.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/conversations.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/conversations.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/agents.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/llm_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/llm_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/llm_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/llm_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/llm_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/llm_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/llm_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/llm_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/llm_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/llm_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/llm_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/llm_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/llm_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/llm_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/llm_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/llm_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/llm_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/llm_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/llm_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/llm_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update docs/source/ar/llm_tutorial.md
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
* Update llm_tutorial.md
* Update _toctree.yml
* Update autoclass_tutorial.md
* Update autoclass_tutorial.md
* Update preprocessing.md
* Update glossary.md
* Update run_scripts.md
* Update run_scripts.md
* Update run_scripts.md
---------
Co-authored-by: Abdullah Mohammed <554032+abodacs@users.noreply.github.com >
2024-09-16 10:02:03 -07:00
Sergio Paniego Blanco
c7a91f5adf
Agents, supercharged - Multi-agents, External tools, and more docs typo fixed (#33478 )
...
* Typo fixed in Agents, supercharged
2024-09-16 18:52:27 +02:00
Merve Noyan
ce62a41880
Add keypoint-detection task guide ( #33274 )
...
---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
2024-09-16 13:08:31 +02:00
Arthur
8bd2b1e8c2
Add support for Pixtral ( #33449 )
...
* initial commit
* gloups
* updates
* work
* weights match
* nits
* nits
* updates to support the tokenizer :)
* updates
* Pixtral processor (#33454 )
* rough outline
* Add in image break and end tokens
* Fix
* Udo some formatting changes
* Set patch_size default
* Fix
* Fix token expansion
* nit in conversion script
* Fix image token list creation
* done
* add expected results
* Process list of list of images (#33465 )
* updates
* working image and processor
* this is the expected format
* some fixes
* push current updated
* working mult images!
* add a small integration test
* Uodate configuration docstring
* Formatting
* Config docstring fix
* simplify model test
* fixup modeling and etests
* Return BatchMixFeature in image processor
* fix some copies
* update
* nits
* Update model docstring
* Apply suggestions from code review
* Fix up
* updates
* revert modeling changes
* update
* update
* fix load safe
* addd liscence
* update
* use pixel_values as required by the model
* skip some tests and refactor
* Add pixtral image processing tests (#33476 )
* Image processing tests
* Add processing tests
* woops
* defaults reflect pixtral image processor
* fixup post merge
* images -> pixel values
* oups sorry Mr docbuilder
* isort
* fix
* fix processor tests
* small fixes
* nit
* update
* last nits
* oups this was really breaking!
* nits
* is composition needs to be true
---------
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com >
2024-09-14 12:28:39 +02:00
Sergio Paniego Blanco
e39b6c1c7c
Corrected Agents and tools documentation links typos ( #33471 )
...
* Corrected agents task link typo
* Corrected chat templating link
* Corrected chat templating link 2
2024-09-13 17:15:20 +02:00
Fanli Lin
a05ce550bf
[docs] refine the doc for train with a script ( #33423 )
...
* add xpu note
* add one more case
* add more
* Update docs/source/en/run_scripts.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
2024-09-12 10:16:12 -07:00
Raushan Turganbay
2f611d30d9
Qwen2-VL: clean-up and add more tests ( #33354 )
...
* clean-up on qwen2-vl and add generation tests
* add video tests
* Update tests/models/qwen2_vl/test_processing_qwen2_vl.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com >
* fix and add better tests
* Update src/transformers/models/qwen2_vl/image_processing_qwen2_vl.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com >
* update docs and address comments
* Update docs/source/en/model_doc/qwen2_vl.md
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com >
* Update docs/source/en/model_doc/qwen2_vl.md
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com >
* update
* remove size at all
---------
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com >
2024-09-12 18:24:04 +02:00
Sergio Paniego Blanco
516ee6adc2
Fix incomplete sentence in Zero-shot object detection documentation ( #33430 )
...
Rephrase sentence in zero-shot object detection docs
2024-09-12 11:25:44 +02:00
Michael Currin
e0ff4321d1
Docs - update formatting of llama3 model card ( #33438 )
...
update formatting of llama3 content
2024-09-12 11:24:56 +02:00
Fanli Lin
cea9ec086a
[docs] add the missing tokenizer when pushing models to huggingface hub ( #33428 )
...
* add tokenizer
* typo
2024-09-11 09:56:55 -07:00
Fanli Lin
c403441339
[docs] add the missing huggingface hub username ( #33431 )
...
* add username
* update username
* add username
2024-09-11 09:56:40 -07:00
Guang Yang
f38590dade
Make StaticCache configurable at model construct time ( #32830 )
...
* Make StaticCache configurable at model construct time
* integrations import structure
* add new doc file to toc
---------
Co-authored-by: Guang Yang <guangyang@fb.com >
Co-authored-by: Joao Gante <joao@huggingface.co >
2024-09-10 16:35:57 +01:00
Alazar
96429e74a8
Add support for GGUF Phi-3 ( #31844 )
...
* Update docs for GGUF supported models
* Add tensor mappings and define class GGUFPhi3Converter
* Fix tokenizer
* Working version
* Attempt to fix some CI failures
* Run ruff format
* Add vocab, merges, decoder methods like LlamaConverter
* Resolve conflicts since Qwen2Moe was added to gguf
- I missed one place when resolving conflict
- I also made a mistake with tests_ggml.py and now has been fixed to reflect
its master version.
2024-09-10 13:32:38 +02:00
Nilay Bhatnagar
eedd21b9e7
Fixed Majority of the Typos in transformers[en] Documentation ( #33350 )
...
* Fixed typo: insted to instead
* Fixed typo: relase to release
* Fixed typo: nighlty to nightly
* Fixed typos: versatible, benchamarks, becnhmark to versatile, benchmark, benchmarks
* Fixed typo in comment: quantizd to quantized
* Fixed typo: architecutre to architecture
* Fixed typo: contibution to contribution
* Fixed typo: Presequities to Prerequisites
* Fixed typo: faste to faster
* Fixed typo: extendeding to extending
* Fixed typo: segmetantion_maps to segmentation_maps
* Fixed typo: Alternativelly to Alternatively
* Fixed incorrectly defined variable: output to output_disabled
* Fixed typo in library name: tranformers.onnx to transformers.onnx
* Fixed missing import: import tensorflow as tf
* Fixed incorrectly defined variable: token_tensor to tokens_tensor
* Fixed missing import: import torch
* Fixed incorrectly defined variable and typo: uromaize to uromanize
* Fixed incorrectly defined variable and typo: uromaize to uromanize
* Fixed typo in function args: numpy.ndarry to numpy.ndarray
* Fixed Inconsistent Library Name: Torchscript to TorchScript
* Fixed Inconsistent Class Name: OneformerProcessor to OneFormerProcessor
* Fixed Inconsistent Class Named Typo: TFLNetForMultipleChoice to TFXLNetForMultipleChoice
* Fixed Inconsistent Library Name Typo: Pytorch to PyTorch
* Fixed Inconsistent Function Name Typo: captureWarning to captureWarnings
* Fixed Inconsistent Library Name Typo: Pytorch to PyTorch
* Fixed Inconsistent Class Name Typo: TrainingArgument to TrainingArguments
* Fixed Inconsistent Model Name Typo: Swin2R to Swin2SR
* Fixed Inconsistent Model Name Typo: EART to BERT
* Fixed Inconsistent Library Name Typo: TensorFLow to TensorFlow
* Fixed Broken Link for Speech Emotion Classification with Wav2Vec2
* Fixed minor missing word Typo
* Fixed minor missing word Typo
* Fixed minor missing word Typo
* Fixed minor missing word Typo
* Fixed minor missing word Typo
* Fixed minor missing word Typo
* Fixed minor missing word Typo
* Fixed minor missing word Typo
* Fixed Punctuation: Two commas
* Fixed Punctuation: No Space between XLM-R and is
* Fixed Punctuation: No Space between [~accelerate.Accelerator.backward] and method
* Added backticks to display model.fit() in codeblock
* Added backticks to display openai-community/gpt2 in codeblock
* Fixed Minor Typo: will to with
* Fixed Minor Typo: is to are
* Fixed Minor Typo: in to on
* Fixed Minor Typo: inhibits to exhibits
* Fixed Minor Typo: they need to it needs
* Fixed Minor Typo: cast the load the checkpoints To load the checkpoints
* Fixed Inconsistent Class Name Typo: TFCamembertForCasualLM to TFCamembertForCausalLM
* Fixed typo in attribute name: outputs.last_hidden_states to outputs.last_hidden_state
* Added missing verbosity level: fatal
* Fixed Minor Typo: take To takes
* Fixed Minor Typo: heuristic To heuristics
* Fixed Minor Typo: setting To settings
* Fixed Minor Typo: Content To Contents
* Fixed Minor Typo: millions To million
* Fixed Minor Typo: difference To differences
* Fixed Minor Typo: while extract To which extracts
* Fixed Minor Typo: Hereby To Here
* Fixed Minor Typo: addition To additional
* Fixed Minor Typo: supports To supported
* Fixed Minor Typo: so that benchmark results TO as a consequence, benchmark
* Fixed Minor Typo: a To an
* Fixed Minor Typo: a To an
* Fixed Minor Typo: Chain-of-though To Chain-of-thought
2024-09-09 10:47:24 +02:00
Aymeric Roucher
489cbfd6d3
Add visit webpage tool ( #33353 )
...
* Add VisitWebpageTool
2024-09-09 10:32:42 +02:00
Wing Lian
62aecd85ff
schedulefree optimizers ( #30079 )
...
* schedulefree optimizers
* fix train instead of eval for optimizer
* fixes and update docs
* chore: lint
* add tests and drop overly-verbose _32bit suffix
* chore: lint
* fix for docs
* fix code review issues
* use duck-typing to avoid per-optimizer patches
* fixup style
* fixup style
* warn if incorrect accelerate version with schedule free
Co-authored-by: Aman Gupta Karmani <aman@tmm1.net >
---------
Co-authored-by: Aman Karmani <aman@tmm1.net >
2024-09-09 09:51:39 +02:00
Nicholas Broad
66bc4def95
add sdpa mbart ( #32033 )
...
* add sdpa mbart
useful for donut
* update sdpa docs
* formatting
* add self._use_sdpa in mbartencoder
* use self.config to check attn
* retrigger checks
* [run-slow] mbart
2024-09-06 17:31:24 -07:00
Daniel Lok
a70286f827
Update author for QLorA/PEFT community notebook ( #33338 )
...
update author
Signed-off-by: Daniel Lok <daniel.lok@databricks.com >
2024-09-06 22:50:26 +02:00
Matt
d7b04ea14d
Fix Prefill docs ( #33352 )
...
last -> final
2024-09-06 17:57:54 +01:00
Ita Zaporozhets
e48e5f1f13
Support reading tiktoken tokenizer.model file ( #31656 )
...
* use existing TikTokenConverter to read tiktoken tokenizer.model file
* del test file
* create titktoken integration file
* adding tiktoken llama test
* ALTNATIVE IMPLEMENTATION: supports llama 405B
* fix one char
* remove redundant line
* small fix
* rm unused import
* flag for converting from tiktokeng
* remove unneeded file
* ruff
* remove llamatiktokenconverter, stick to general converter
* tiktoken support v2
* update test
* remove stale changes
* udpate doc
* protect import
* use is_protobuf_available
* add templateprocessor in tiktokenconverter
* reverting templateprocessor from tiktoken support
* update test
* add require_tiktoken
* dev-ci
* trigger build
* trigger build again
* dev-ci
* [build-ci-image] tiktoken
* dev-ci
* dev-ci
* dev-ci
* dev-ci
* change tiktoken file name
* feedback review
* feedback rev
* applying feedback, removing tiktoken converters
* conform test
* adding docs for review
* add doc file for review
* add doc file for review
* add doc file for review
* support loading model without config.json file
* Revert "support loading model without config.json file"
This reverts commit 2753602e51c34cef2f184eb11f36d2ad1b02babb.
* remove dev var
* updating docs
* safely import protobuf
* fix protobuf import error
* fix protobuf import error
* trying isort to fix ruff error
* fix ruff error
* try to fix ruff again
* try to fix ruff again
* try to fix ruff again
* doc table of contents
* add fix for consistency.dockerfile torchaudio
* ruff
* applying feedback
* minor typo
* merging with push-ci-image
* clean up imports
* revert dockerfile consistency
2024-09-06 14:24:02 +02:00
Joao Gante
2b789f27f3
Docs: add more cross-references to the KV cache docs ( #33323 )
...
* add more cross-references
* nit
* import guard
* more import guards
* nit
* Update src/transformers/generation/configuration_utils.py
2024-09-06 10:22:00 +01:00
Daniel Lok
5792c459ed
Add a community notebook for fine-tuning with QLoRA, PEFT, and MLflow ( #33319 )
...
add notebook for finetuning with mlflow
Signed-off-by: Daniel Lok <daniel.lok@databricks.com >
2024-09-06 09:35:01 +02:00
Vladislav Bronzov
5d11de4a2f
Add Qwen2Moe GGUF loading support ( #33264 )
...
* update gguf doc, config and tensor mapping
* add qwen2moe architecture support, GGUFQwen2MoeConverter and q4 unit tests
* apply code style fixes
* reformat files
* assign GGUFQwen2Converter to qwen2_moe
2024-09-05 17:42:03 +02:00
Niklas Muennighoff
03164ba14e
Add paper link ( #33305 )
2024-09-05 15:49:28 +02:00
Raushan Turganbay
43df47d8e7
Llava Onevision: add model ( #32673 )
...
* working version
* fix copies
* update
* tests
* update docs
* codestyle
* add more tests
* add returns for docs
* clean up
* Update src/transformers/models/llava_onevision/processing_llava_onevision.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com >
* updates
* codestyle
* style
* shouldn't be reversed
* [run-slow] llava_onevision
* [run-slow] llava_onevision
* add pooling in videos
* [run-slow] llava_onevision
* num-logits-to-keep
* [run-slow] llava_onevision
* [run-slow] llava_onevision
* Update tests/test_modeling_common.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com >
* video matched orig impl
* fix tests
* chat template was modified
* Update docs/source/en/model_doc/llava_onevision.md
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com >
* add morer info in the doc page
---------
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com >
2024-09-05 14:43:20 +05:00
Aymeric Roucher
cfd92c64f5
Add new documentation page for advanced agent usage ( #33265 )
...
* Add new documentation page for advanced agent usage
2024-09-04 18:19:54 +02:00
Matt
01c8c6c419
Add a warning to the chat template docs about the tool_calls format ( #33277 )
...
* Add a warning to the chat template docs
* Add a warning to the chat template docs
* Add a warning to the chat template docs
2024-09-04 17:13:34 +01:00
Raushan Turganbay
ebbe8d8014
Cache docs: update ( #32929 )
...
* some changes
* more updates
* fix cache copy
* nits
* nits
* add tests
2024-09-04 15:05:31 +05:00
Niklas Muennighoff
ecd61c6286
Add OLMoE ( #32406 )
...
* Add OLMoE
* Add OLMoE
* Updates
* Make norm optional; add keys
* Add output
* Add
* Fix dtype
* Fix eos config
* Update
* Add OLMoE
* Fix OLMoE path
* Format
* Format
* Rmv copy statement
* Rmv copy statement
* Format
* Add copies
* Cp rotary
* Fix aming
* Fix naming
* Update RoPE integration; num_logits_to_keep; Add copy statements
* Add eps to config
* Format
* Add aux loss
* Adapt router_aux_loss_coef
* Update md
* Adapt
* adapt tests
2024-09-03 18:43:12 +02:00
Omar Salman
03c12d0d63
Add sdpa support for Albert ( #32092 )
...
* Add sdpa support for Albert
* [run_slow] albert
* Add benchmarks and PR suggestion
* Fix quality
* Fix
* [run_slow] albert
2024-09-03 14:01:00 +01:00
Matt
0d86727354
Update chat template docs to remove Blenderbot ( #33254 )
...
* Update docs to remove obsolete Blenderbot
* Remove another reference to Blenderbot
2024-09-03 12:18:04 +01:00
Isotr0py
edeca4387c
🚨 Support dequantization for most GGML types ( #32625 )
...
* use gguf internal dequantize
* add Q5_0 test
* add iq1 test
* add remained test
* remove duplicated test
* update docs
* add gguf version limit
* make style
* update gguf import catch
* revert vocab_size patch
* make style
* use GGUF_MIN_VERSION everywhere
2024-09-03 12:58:14 +02:00
Sergio Paniego Blanco
28952248b1
Fixed typo repeated word in DETR docs ( #33250 )
2024-09-02 17:19:18 +02:00
Matt
52a0213755
Add assistant prefill for chat templates and TextGenerationPipeline ( #33198 )
...
* Add assistant prefill to chat templates
* Add assistant prefill to pipeline
* Add assistant prefill to pipeline
* Tweak another test that ended in assistant message
* Update tests that ended in assistant messages
* Update tests that ended in assistant messages
* Replace assistant_prefill with continue_final_message
* Allow passing continue_final_message to pipeline
* Small fixup
* Add continue_final_message as a pipeline kwarg
* Update docstrings
* Move repos to hf-internal-testing!
* Update src/transformers/tokenization_utils_base.py
Co-authored-by: Lysandre Debut <hi@lysand.re >
* Add explanatory comment
* make fixup
* Update chat templating docs to explain continue_last_message
---------
Co-authored-by: Lysandre Debut <hi@lysand.re >
2024-09-02 13:23:47 +01:00
Aymeric Roucher
1ca9ff5c91
Add duckduckgo search tool ( #32882 )
...
* Add duckduckgo search tool
2024-09-02 09:56:20 +02:00
Merve Noyan
2e3f8f7474
Add video text to text docs ( #33164 )
...
---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
2024-09-01 12:06:31 +03:00
