SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity
Files
bf64b8cf095a23303315e1347d8eac0bce9d73be
HuggingFace_transformer/tests/test_tokenization_xlm.py
Lysandre Debut 8b63a01d95 XLM tokenizer should encode with bos token (#3791) 
* XLM tokenizer should encode with bos token

* Update tests
2020-04-17 11:28:55 -04:00

3.2 KiB
Raw Blame History

View Raw
Reference in New Issue View Git Blame Copy Permalink