SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity

Fix attn mask ignore logic in training-time trace (#32613)

Browse Source 
* fix attn mask logic for training-time trace

* add test

* fix

* fix

* fix

* fix

* fix

* format

* [run-slow] llama

* avoid accelearate

* [run-slow] llama
This commit is contained in:
Longjie Zheng
2024-10-04 13:00:45 -04:00
committed by GitHub
parent 614660fdb9
commit 0d1692a49b
7 changed files with 55 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
tests/models/mistral/test_modeling_mistral.py
View File

@@ -677,7 +677,7 @@ class MistralIntegrationTest(unittest.TestCase):
tokenizer = AutoTokenizer.from_pretrained("mistralai/Mistral-7B-v0.1", use_fast=False)
tokenizer.pad_token = tokenizer.eos_token
model = MistralForCausalLM.from_pretrained(
"mistralai/Mistral-7B-v0.1", device_map="sequential", torch_dtype=torch.float16
"mistralai/Mistral-7B-v0.1", device_map=torch_device, torch_dtype=torch.float16
)
inputs = tokenizer(prompts, return_tensors="pt", padding=True).to(model.device)