SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity
Files
f54dc3f4d5517ff0173cd505cf0e2c7569a61835
HuggingFace_transformer/tests/test_tokenization_xlm.py
Lysandre Debut 8b63a01d95 XLM tokenizer should encode with bos token (#3791) 
* XLM tokenizer should encode with bos token

* Update tests
2020-04-17 11:28:55 -04:00

3.2 KiB
Raw Blame History

View Raw
Reference in New Issue View Git Blame Copy Permalink