SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity
Files
c63fcabfe92229eac983ce743f4e9fce863c84dd
HuggingFace_transformer/tests/test_tokenization_auto.py
Sylvain Gugger c4ecd234f2 Fix AutoTokenizer when no fast tokenizer is available (#13336) 
* Fix AutoTokenizer when a tokenizer has no fast version

* Add test
2021-08-30 11:55:18 -04:00

8.5 KiB
Raw Blame History

View Raw
Reference in New Issue View Git Blame Copy Permalink