[docs] Increase visibility of torch_dtype="auto" (#35067)

* auto-dtype * feedback
2024-12-04 09:18:44 -08:00
parent baa3b22137
commit 1ed1de2fec
11 changed files with 49 additions and 35 deletions
--- a/docs/source/en/perf_infer_cpu.md
+++ b/docs/source/en/perf_infer_cpu.md
@@ -41,7 +41,7 @@ Enable BetterTransformer with the [`PreTrainedModel.to_bettertransformer`] metho
 ```py
 from transformers import AutoModelForCausalLM

-model = AutoModelForCausalLM.from_pretrained("bigcode/starcoder")
+model = AutoModelForCausalLM.from_pretrained("bigcode/starcoder", torch_dtype="auto")
 ```

 ## TorchScript