Sylvain Gugger adb70eda4d AutoTokenizer: infer the class from the tokenizer config if possible (#12208) ...

* AutoTokenizer: infer the class from the tokenizer config if possible

* Add tests

* Update src/transformers/models/auto/tokenization_auto.py

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

2021-06-17 12:39:22 -04:00

7.4 KiB

Raw Blame History

View Raw