SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity

Fixup mapping

Browse Source
This commit is contained in:
Julien Chaumond
2020-01-13 23:34:19 +00:00
parent b20f11d4ca
commit 894812c652
1 changed files with 2 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
src/transformers/tokenization_auto.py
View File

@@ -60,8 +60,8 @@ TOKENIZER_MAPPING = OrderedDict(
(DistilBertConfig, DistilBertTokenizer), (DistilBertConfig, DistilBertTokenizer),
(AlbertConfig, AlbertTokenizer), (AlbertConfig, AlbertTokenizer),
(CamembertConfig, CamembertTokenizer), (CamembertConfig, CamembertTokenizer),
(RobertaConfig, XLMRobertaTokenizer), (RobertaConfig, RobertaTokenizer),
(XLMRobertaConfig, RobertaTokenizer), (XLMRobertaConfig, XLMRobertaTokenizer),
(BertConfig, BertTokenizer), (BertConfig, BertTokenizer),
(OpenAIGPTConfig, OpenAIGPTTokenizer), (OpenAIGPTConfig, OpenAIGPTTokenizer),
(GPT2Config, GPT2Tokenizer), (GPT2Config, GPT2Tokenizer),