Stas Bekman 40d60e1536 fix tokenizer_class_from_name for models with - in the name (#13251) ...

* fix tokenizer_class_from_name

* Update src/transformers/models/auto/tokenization_auto.py

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

* add test

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

2021-08-26 04:29:14 -04:00

8.2 KiB

Raw Blame History

View Raw