SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity

[Follow up 213]

Browse Source 
Masked indices should have -1 and not -100. Updating documentation + scripts that were forgotten
This commit is contained in:
Lysandre
2020-02-03 16:08:05 -05:00
parent 522c5b5533
commit 239dd23f64
14 changed files with 37 additions and 37 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
src/transformers/modeling_ctrl.py
View File

@@ -479,8 +479,8 @@ class CTRLLMHeadModel(CTRLPreTrainedModel):
labels (:obj:`torch.LongTensor` of shape :obj:`(batch_size, sequence_length)`, `optional`, defaults to :obj:`None`):
Labels for language modeling.
Note that the labels **are shifted** inside the model, i.e. you can set ``lm_labels = input_ids``
Indices are selected in ``[-100, 0, ..., config.vocab_size]``
All labels set to ``-100`` are ignored (masked), the loss is only
Indices are selected in ``[-1, 0, ..., config.vocab_size]``
All labels set to ``-1`` are ignored (masked), the loss is only
computed for labels in ``[0, ..., config.vocab_size]``
Return: