Jerry Zhang 2af272c101 Add autoquant support for torchao quantizer (#35503) ...

* Add autoquant support for torchao quantizer

Summary:
att, also verified that autoquantized model can be saved and loaded:

save: https://gist.github.com/jerryzh168/01d367aaf44dbbbfd4068a4a10a00061
load: https://gist.github.com/jerryzh168/d5c6c401b2abdf18e0b6771341f1525c

Test Plan:
tested locally with above script
model uploaded to https://huggingface.co/jerryzh168/llama3-8b-autoquant

Reviewers:

Subscribers:

Tasks:

Tags:

* add test

* ruff fix

* ruff reformat

* add docs and min_sqnr support

* format

* format

* fix test

* update doc

* format

* remove disable_compile

* format

2025-02-24 15:54:16 +01:00

..

internal

Implement AsyncTextIteratorStreamer for asynchronous streaming (#34931)

2024-12-20 12:08:12 +01:00

main_classes

Efficient Inference Kernel for SpQR (#34976)

2025-02-13 16:22:58 +01:00

model_doc

Add SigLIP 2 (#36323)

2025-02-21 09:04:19 +00:00

quantization

Add autoquant support for torchao quantizer (#35503)

2025-02-24 15:54:16 +01:00

tasks

Move DataCollatorForMultipleChoice from the docs to the package (#34763)

2025-02-13 12:01:28 +01:00

_config.py

Add optimized PixtralImageProcessorFast (#34836)

2024-11-28 16:04:05 +01:00

_redirects.yml

Docs / Quantization: Redirect deleted page (#31063)

2024-05-28 18:29:22 +02:00

_toctree.yml

Add SigLIP 2 (#36323)

2025-02-21 09:04:19 +00:00

accelerate.md

Fixed Majority of the Typos in transformers[en] Documentation (#33350)

2024-09-09 10:47:24 +02:00

add_new_model.md

Model addition timeline (#33762)

2024-09-27 17:15:13 +02:00

add_new_pipeline.md

[docs] Follow up register_pipeline (#35310)

2024-12-20 09:22:44 -08:00

agents_advanced.md

[doctest] Fixes (#35863)

2025-01-26 15:26:38 -08:00

agents.md

Multiple typo fixes in Tutorials docs (#35035)

2024-12-02 15:26:34 +00:00

attention.md

[Docs] Fix broken links and syntax issues (#28918)

2024-02-08 14:13:35 -08:00

autoclass_tutorial.md

[docs] Increase visibility of torch_dtype="auto" (#35067)

2024-12-04 09:18:44 -08:00

bertology.md

Fixed Majority of the Typos in transformers[en] Documentation (#33350)

2024-09-09 10:47:24 +02:00

big_models.md

[docs] Big model loading (#29920)

2024-04-01 18:47:32 -07:00

chat_template_advanced.md

Chat template docs (#36163)

2025-02-14 10:32:14 +01:00

chat_template_basics.md

Chat template docs (#36163)

2025-02-14 10:32:14 +01:00

chat_template_multimodal.md

Chat template docs (#36163)

2025-02-14 10:32:14 +01:00

chat_template_tools_and_documents.md

Chat template docs (#36163)

2025-02-14 10:32:14 +01:00

community.md

Fixed Majority of the Typos in transformers[en] Documentation (#33350)

2024-09-09 10:47:24 +02:00

contributing.md

Enable doc in Spanish (#16518)

2022-04-04 10:25:46 -04:00

conversations.md

[docs] change temperature to a positive value (#32077)

2024-07-23 17:47:51 +01:00

create_a_model.md

Enable HF pretrained backbones (#31145)

2024-06-06 22:02:38 +01:00

custom_models.md

Updated the custom_models.md changed cross_entropy code (#33118)

2024-08-26 13:15:43 +02:00

debugging.md

DeepSpeed github repo move sync (#36021)

2025-02-05 08:19:31 -08:00

deepspeed.md

DeepSpeed github repo move sync (#36021)

2025-02-05 08:19:31 -08:00

fast_tokenizers.md

Migrate doc files to Markdown. (#24376)

2023-06-20 18:07:47 -04:00

fsdp.md

Fix docs typos. (#35465)

2025-01-02 11:29:46 +01:00

generation_strategies.md

[doctest] Fixes (#35863)

2025-01-26 15:26:38 -08:00

gguf.md

Add Gemma2 GGUF support (#34002)

2025-01-03 14:50:07 +01:00

glossary.md

Fix typos (#31819)

2024-07-08 11:52:47 +01:00

how_to_hack_models.md

Add utility for Reload Transformers imports cache for development workflow #35508 (#35858)

2025-02-12 12:45:11 +01:00

hpo_train.md

Trainer - deprecate tokenizer for processing_class (#32385)

2024-10-02 14:08:46 +01:00

index.md

Add SigLIP 2 (#36323)

2025-02-21 09:04:19 +00:00

installation.md

[docs] uv install (#35821)

2025-01-27 08:49:28 -08:00

kv_cache.md

[docs] no hard-coding cuda (#36043)

2025-02-05 08:22:33 -08:00

llm_optims.md

Update llm_optims docs for sdpa_kernel (#35481)

2025-01-06 08:54:31 -08:00

llm_tutorial_optimization.md

feat: add support for tensor parallel training workflow with accelerate (#34194)

2025-02-18 14:05:46 +01:00

llm_tutorial.md

[docs] no hard coding cuda as bnb has multi-backend support (#35867)

2025-02-05 08:20:02 -08:00

model_memory_anatomy.md

Enable BNB multi-backend support (#31098)

2024-09-24 03:40:56 -06:00

model_sharing.md

[docs] update not-working model revision (#34682)

2024-11-11 07:09:31 -08:00

model_summary.md

model_summary.md - Restore link to Harvard's Annotated Transformer. (#29702)

2024-03-23 18:29:39 -07:00

modular_transformers.md

Improve modular documentation (#35737)

2025-01-21 17:53:30 +01:00

multilingual.md

Update all references to canonical models (#29001)

2024-02-16 08:16:58 +01:00

notebooks.md

Enable doc in Spanish (#16518)

2022-04-04 10:25:46 -04:00

pad_truncation.md

Fixed Majority of the Typos in transformers[en] Documentation (#33350)

2024-09-09 10:47:24 +02:00

peft.md

Fixed Majority of the Typos in transformers[en] Documentation (#33350)

2024-09-09 10:47:24 +02:00

perf_hardware.md

Fixed Majority of the Typos in transformers[en] Documentation (#33350)

2024-09-09 10:47:24 +02:00

perf_infer_cpu.md

[docs] Increase visibility of torch_dtype="auto" (#35067)

2024-12-04 09:18:44 -08:00

perf_infer_gpu_multi.md

Update doc re list of models supporting TP (#35864)

2025-02-12 15:53:27 +01:00

perf_infer_gpu_one.md

Add SigLIP 2 (#36323)

2025-02-21 09:04:19 +00:00

perf_torch_compile.md

[docs] use device-agnostic instead of cuda (#35047)

2024-12-03 10:53:45 -08:00

perf_train_cpu_many.md

[doc] use full path for run_qa.py (#34914)

2024-11-26 09:23:44 -08:00

perf_train_cpu.md

[doc] use full path for run_qa.py (#34914)

2024-11-26 09:23:44 -08:00

perf_train_gpu_many.md

feat: add support for tensor parallel training workflow with accelerate (#34194)

2025-02-18 14:05:46 +01:00

perf_train_gpu_one.md

layernorm_decay_fix (#35927)

2025-02-04 11:01:49 +01:00

perf_train_special.md

Update all references to canonical models (#29001)

2024-02-16 08:16:58 +01:00

perf_train_tpu_tf.md

Fixed Majority of the Typos in transformers[en] Documentation (#33350)

2024-09-09 10:47:24 +02:00

performance.md

Simplify Tensor Parallel implementation with PyTorch TP (#34184)

2024-11-18 19:51:49 +01:00

perplexity.md

[docs] use device-agnostic API instead of cuda (#34913)

2024-11-26 09:23:34 -08:00

philosophy.md

[docs] fixed links with 404 (#27327)

2023-11-06 19:45:03 +00:00

pipeline_tutorial.md

[docs] Increase visibility of torch_dtype="auto" (#35067)

2024-12-04 09:18:44 -08:00

pipeline_webserver.md

Update all references to canonical models (#29001)

2024-02-16 08:16:58 +01:00

pr_checks.md

Fixed Majority of the Typos in transformers[en] Documentation (#33350)

2024-09-09 10:47:24 +02:00

preprocessing.md

Fixed Majority of the Typos in transformers[en] Documentation (#33350)

2024-09-09 10:47:24 +02:00

quicktour.md

[chat] docs fix (#35840)

2025-01-22 14:32:27 +00:00

run_scripts.md

[docs] refine the doc for train with a script (#33423)

2024-09-12 10:16:12 -07:00

sagemaker.md

Fixed Majority of the Typos in transformers[en] Documentation (#33350)

2024-09-09 10:47:24 +02:00

serialization.md

[docs] fix model checkpoint name (#36075)

2025-02-07 12:41:52 -08:00

task_summary.md

[doctest] Fixes (#35863)

2025-01-26 15:26:38 -08:00

tasks_explained.md

fix: Wrong task mentioned in docs (#34757)

2024-11-18 18:42:28 +00:00

testing.md

[tests] add XPU part to testing (#34778)

2024-11-18 09:59:11 -08:00

tf_xla.md

fix(docs): Fixed a link in docs (#32274)

2024-07-29 10:50:43 +01:00

tflite.md

Update all references to canonical models (#29001)

2024-02-16 08:16:58 +01:00

tiktoken.md

Updated documentation and added conversion utility (#34319)

2024-11-25 18:44:09 +01:00

tokenizer_summary.md

[docs] Spanish translation of tokenizer_summary.md (#31154)

2024-06-03 16:52:23 -07:00

torchscript.md

Fixed Majority of the Typos in transformers[en] Documentation (#33350)

2024-09-09 10:47:24 +02:00

trainer.md

feat: add support for tensor parallel training workflow with accelerate (#34194)

2025-02-18 14:05:46 +01:00

training.md

[docs] Increase visibility of torch_dtype="auto" (#35067)

2024-12-04 09:18:44 -08:00

troubleshooting.md

Update all references to canonical models (#29001)

2024-02-16 08:16:58 +01:00