SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity

[from_pretrained] Allow tokenizer_type ≠ model_type (#6995)

Browse Source
This commit is contained in:
Julien Chaumond
2020-09-09 10:22:59 +02:00
committed by GitHub
parent 03e363f9ae
commit ed71c21d6a
4 changed files with 28 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
src/transformers/testing_utils.py
View File

@@ -15,6 +15,7 @@ from .file_utils import _tf_available, _torch_available, _torch_tpu_available
SMALL_MODEL_IDENTIFIER = "julien-c/bert-xsmall-dummy"
DUMMY_UNKWOWN_IDENTIFIER = "julien-c/dummy-unknown"
DUMMY_DIFF_TOKENIZER_IDENTIFIER = "julien-c/dummy-diff-tokenizer"
# Used to test Auto{Config, Model, Tokenizer} model_type detection.