Files
HuggingFace_transformer/docs/source
Oren Amsalem d24ea708d7 Actually the extra_id are from 0-99 and not from 1-100 (#5967)
a = tokenizer.encode("we got a <extra_id_99>", return_tensors='pt',add_special_tokens=True)
print(a)
>tensor([[   62,   530,     3,     9, 32000]])
a = tokenizer.encode("we got a <extra_id_100>", return_tensors='pt',add_special_tokens=True)
print(a)
>tensor([[   62,   530,     3,     9,     3,     2, 25666,   834,    23,    26,
           834,  2915,  3155]])
2020-07-30 06:13:29 -04:00
..
2020-07-24 09:16:28 -04:00
2020-07-24 09:16:28 -04:00
2020-06-29 16:08:57 +02:00
2020-07-06 18:49:44 -04:00
2020-07-02 09:19:33 -04:00
2020-07-29 11:02:35 +02:00
2020-06-24 07:56:14 -04:00
2020-04-06 14:32:39 -04:00
2020-07-14 10:39:44 -06:00
2020-06-24 07:56:14 -04:00
2020-07-08 19:52:44 -04:00