SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity

fixed small typo in code example (#22982)

Browse Source 
fixed typo in code example

fixed a really small typo in the docs of single gpu inference
This commit is contained in:
Jari Van Melckebeke
2023-04-25 14:56:21 +02:00
committed by GitHub
parent 0a570dbd2e
commit 81c1910c86
1 changed files with 2 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
docs/source/en/perf_infer_gpu_one.mdx
View File

@@ -71,7 +71,7 @@ model_name = "bigscience/bloom-2b5"
tokenizer = AutoTokenizer.from_pretrained(model_name) tokenizer = AutoTokenizer.from_pretrained(model_name)
model_8bit = AutoModelForCausalLM.from_pretrained(model_name, device_map="auto", load_in_8bit=True) model_8bit = AutoModelForCausalLM.from_pretrained(model_name, device_map="auto", load_in_8bit=True)
text = "Hello, my llama is cute" prompt = "Hello, my llama is cute"
inputs = tokenizer(prompt, return_tensors="pt").to("cuda") inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
generated_ids = model.generate(**inputs) generated_ids = model.generate(**inputs)
outputs = tokenizer.batch_decode(generated_ids, skip_special_tokens=True) outputs = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)