Files
HuggingFace_transformer/docs/source/model_doc
Oren Amsalem d24ea708d7 Actually the extra_id are from 0-99 and not from 1-100 (#5967)
a = tokenizer.encode("we got a <extra_id_99>", return_tensors='pt',add_special_tokens=True)
print(a)
>tensor([[   62,   530,     3,     9, 32000]])
a = tokenizer.encode("we got a <extra_id_100>", return_tensors='pt',add_special_tokens=True)
print(a)
>tensor([[   62,   530,     3,     9,     3,     2, 25666,   834,    23,    26,
           834,  2915,  3155]])
2020-07-30 06:13:29 -04:00
..
2020-07-10 17:31:02 -04:00
2020-06-08 11:28:19 -04:00
2020-06-08 11:28:19 -04:00
2020-07-10 17:31:02 -04:00
2020-07-10 17:31:02 -04:00
2020-07-10 17:31:02 -04:00
2020-07-10 17:31:02 -04:00
2020-06-08 11:28:19 -04:00
2020-06-08 11:28:19 -04:00
2020-07-10 17:31:02 -04:00
2020-06-16 16:36:58 -04:00
2020-06-08 11:28:19 -04:00
2020-07-10 17:31:02 -04:00
2020-07-10 17:31:02 -04:00