SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity

Fix DataCollatorForLanguageModeling (#8621)

Browse Source
This commit is contained in:
Sylvain Gugger
2020-11-18 10:02:50 -05:00
committed by GitHub
parent f6fe41c96b
commit 8fcb6935a1
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
src/transformers/data/data_collator.py
View File

@@ -261,7 +261,7 @@ class DataCollatorForLanguageModeling:
batch["input_ids"], special_tokens_mask=special_tokens_mask batch["input_ids"], special_tokens_mask=special_tokens_mask
) )
else: else:
labels = batch["input_ids"] labels = batch["input_ids"].clone()
if self.tokenizer.pad_token_id is not None: if self.tokenizer.pad_token_id is not None:
labels[labels == self.tokenizer.pad_token_id] = -100 labels[labels == self.tokenizer.pad_token_id] = -100
batch["labels"] = labels batch["labels"] = labels