eustlb 967045082f Add voxtral (#39429) ...

* draft

* draft update (conversion working)

* mend

* draft update

* draft update: working generate

* refactor

* VoxtralProcessor draft

* processor update

* update convert_tekken_tokenizer

* refactor processor

* update convert

* make style

* better handle prefil

* make style

* add tests

* add mistral_common audio loading

* processor update

* revert changes

* audio utils update

* add audio to apply chat template mistral update

* voxtral processor update

* fix

* udpate converstion script

* make mistral tokenier from pretrain work from local dir

* fix udpates

* add integration tests

* add batched version

* processor docstring

* make style

* revert convert_tekken_tokenizer changes

* revert processing_qwen2.5 changes

* add multi-turn test

* processor improvements

* address review changes

* Update src/transformers/tokenization_mistral_common.py

Co-authored-by: Julien Denize <40604584+juliendenize@users.noreply.github.com>

* update audio utils

* nits

* integration test update

* correct _support

* update tests

* test update

* update integration tests

* fix

* fix

* fix

* add test_apply_chat_template_with_audio

* add model doc

* model doc

* nit

* doc uptade

* nit

* processor improvement

* ensure default is 3B

* nits

* make

* make

* convert modular

* update checkpoint

* fix test

* make

* make

* autos

* make

* make

* nit

* nit

* nit

---------

Co-authored-by: Julien Denize <40604584+juliendenize@users.noreply.github.com>
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>

2025-07-18 00:02:04 +00:00

..

bettertransformer

Use Python 3.9 syntax in tests (#37343)

2025-04-08 14:12:08 +02:00

commands

[serve] Add speech to text (/v1/audio/transcriptions) (#39434)

2025-07-17 14:29:57 +00:00

deepspeed

Remove script datasets in tests (#38940)

2025-06-25 14:31:20 +00:00

extended

Add Optional to remaining types (#37808)

2025-04-28 14:20:45 +01:00

fixtures

[tests] remove tests from libraries with deprecated support (flax, tensorflow_text, ...) (#39051)

2025-06-26 16:25:00 +01:00

fsdp

Gaudi3 CI (#38790)

2025-06-23 10:56:51 +02:00

generation

[cache] make all classes cache compatible finally (#38635)

2025-07-16 14:00:17 +02:00

models

Add voxtral (#39429)

2025-07-18 00:02:04 +00:00

optimization

[tests] remove TF tests (uses of require_tf) (#38944)

2025-06-25 17:29:10 +00:00

peft_integration

Enable some ruff checks for performance and readability (#39383)

2025-07-17 13:21:59 +00:00

pipelines

fix(pipelines): QA pipeline returns fewer than top_k results in batch mode (#39193)

2025-07-17 10:24:30 +02:00

quantization

Remove residual quantization attribute from dequantized models (#39373)

2025-07-15 17:16:10 +02:00

repo_utils

[modular] speedup check_modular_conversion with multiprocessing (#37456)

2025-07-10 19:07:59 +01:00

sagemaker

Deprecate TF + JAX (#38758)

2025-06-11 17:28:06 +01:00

tensor_parallel

enable static cache on TP model (#39164)

2025-07-09 21:14:45 +00:00

tokenization

[tests] remove tests from libraries with deprecated support (flax, tensorflow_text, ...) (#39051)

2025-06-26 16:25:00 +01:00

trainer

Fix tests due to breaking change in accelerate (#39451)

2025-07-17 13:51:50 +01:00

utils

Fix typing order (#39467)

2025-07-17 15:47:31 +00:00

__init__.py

…

causal_lm_tester.py

Refactor DBRX tests to use CausalLMModelTest base classes (#38475)

2025-06-13 16:22:12 +01:00

test_backbone_common.py

Use Python 3.9 syntax in tests (#37343)

2025-04-08 14:12:08 +02:00

test_configuration_common.py

Update composition flag usage (#36263)

2025-04-09 11:48:49 +02:00

test_feature_extraction_common.py

Use Python 3.9 syntax in tests (#37343)

2025-04-08 14:12:08 +02:00

test_image_processing_common.py

Fix overriding Fast Image/Video Processors instance attributes affect other instances (#39363)

2025-07-12 23:39:06 +00:00

test_image_transforms.py

[tests] remove tests from libraries with deprecated support (flax, tensorflow_text, ...) (#39051)

2025-06-26 16:25:00 +01:00

test_modeling_common.py

Enable some ruff checks for performance and readability (#39383)

2025-07-17 13:21:59 +00:00

test_pipeline_mixin.py

Enable some ruff checks for performance and readability (#39383)

2025-07-17 13:21:59 +00:00

test_processing_common.py

[chat template] add a testcase for kwargs (#39415)

2025-07-16 11:31:35 +02:00

test_sequence_feature_extraction_common.py

[tests] remove TF tests (uses of require_tf) (#38944)

2025-06-25 17:29:10 +00:00

test_tokenization_common.py

[tests] remove tests from libraries with deprecated support (flax, tensorflow_text, ...) (#39051)

2025-06-26 16:25:00 +01:00

test_tokenization_mistral_common.py

Add voxtral (#39429)

2025-07-18 00:02:04 +00:00

test_training_args.py

Fix TrainingArguments.torch_empty_cache_steps post_init check (#36734)

2025-03-17 16:09:46 +01:00

test_video_processing_common.py

Fix overriding Fast Image/Video Processors instance attributes affect other instances (#39363)

2025-07-12 23:39:06 +00:00