Commit Graph

10 Commits

Author SHA1 Message Date
Lysandre
74d0bcb6ff Fix special tokens addition in decoder 2019-11-12 15:27:57 -05:00
Lysandre
7d709e55ed Remove 2019-10-22 14:12:33 -04:00
thomwolf
78ef1a9930 fixes 2019-10-04 17:59:44 -04:00
thomwolf
6c1d0bc066 update encode_plus - add truncation strategies 2019-10-04 17:38:38 -04:00
LysandreJik
aebd83230f Update naming + remove f string in run_lm_finetuning example 2019-10-03 11:31:36 -04:00
LysandreJik
651bfb7ad5 always_truncate by default 2019-10-03 11:31:36 -04:00
LysandreJik
cc412edd42 Supports already existing special tokens 2019-10-03 11:31:36 -04:00
LysandreJik
2f259b228e Sequence IDS 2019-10-03 11:31:36 -04:00
LysandreJik
7c789c337d Always truncate argument in the encode method 2019-10-03 11:31:36 -04:00
thomwolf
31c23bd5ee [BIG] pytorch-transformers => transformers 2019-09-26 10:15:53 +02:00