SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity
Files
835b76a46f37ecaa00eebf6d7f190297b44d6103
HuggingFace_transformer/src/transformers
History
Anthony MOI 835b76a46f Handle unk_token 
As we discussed, this is handled here directly 
cc @thomwolf
2019-12-26 14:42:55 -05:00
..
commands
Improve exception type.
2019-12-23 21:27:38 +01:00
data
Improve exception type.
2019-12-23 21:27:38 +01:00
__init__.py
BertTokenizerFast
2019-12-24 12:24:24 -05:00
__main__.py
Move source code inside a src subdirectory.
2019-12-22 14:15:13 +01:00
configuration_albert.py
Move source code inside a src subdirectory.
2019-12-22 14:15:13 +01:00
configuration_auto.py
Remove __future__ imports.
2019-12-22 17:47:54 +01:00
configuration_bert.py
Remove __future__ imports.
2019-12-22 17:47:54 +01:00
configuration_camembert.py
Remove __future__ imports.
2019-12-22 17:47:54 +01:00
configuration_ctrl.py
Remove __future__ imports.
2019-12-22 17:47:54 +01:00
configuration_distilbert.py
Remove __future__ imports.
2019-12-22 17:47:54 +01:00
configuration_gpt2.py
Remove __future__ imports.
2019-12-22 17:47:54 +01:00
configuration_mmbt.py
Remove __future__ imports.
2019-12-22 17:47:54 +01:00
configuration_openai.py
Remove __future__ imports.
2019-12-22 17:47:54 +01:00
configuration_roberta.py
Remove __future__ imports.
2019-12-22 17:47:54 +01:00
configuration_t5.py
Remove __future__ imports.
2019-12-22 17:47:54 +01:00
configuration_transfo_xl.py
Remove __future__ imports.
2019-12-22 17:47:54 +01:00
configuration_utils.py
Use built-in open().
2019-12-22 18:38:56 +01:00
configuration_xlm_roberta.py
Remove __future__ imports.
2019-12-22 17:47:54 +01:00
configuration_xlm.py
Remove __future__ imports.
2019-12-22 17:47:54 +01:00
configuration_xlnet.py
Remove __future__ imports.
2019-12-22 17:47:54 +01:00
convert_albert_original_tf_checkpoint_to_pytorch.py
Remove __future__ imports.
2019-12-22 17:47:54 +01:00
convert_bert_original_tf_checkpoint_to_pytorch.py
Remove __future__ imports.
2019-12-22 17:47:54 +01:00
convert_bert_pytorch_checkpoint_to_original_tf.py
Move source code inside a src subdirectory.
2019-12-22 14:15:13 +01:00
convert_gpt2_original_tf_checkpoint_to_pytorch.py
Use built-in open().
2019-12-22 18:38:56 +01:00
convert_openai_original_tf_checkpoint_to_pytorch.py
Use built-in open().
2019-12-22 18:38:56 +01:00
convert_pytorch_checkpoint_to_tf2.py
Remove __future__ imports.
2019-12-22 17:47:54 +01:00
convert_roberta_original_pytorch_checkpoint_to_pytorch.py
Remove __future__ imports.
2019-12-22 17:47:54 +01:00
convert_t5_original_tf_checkpoint_to_pytorch.py
Remove __future__ imports.
2019-12-22 17:47:54 +01:00
convert_transfo_xl_original_tf_checkpoint_to_pytorch.py
Use built-in open().
2019-12-22 18:38:56 +01:00
convert_xlm_original_pytorch_checkpoint_to_pytorch.py
Use built-in open().
2019-12-22 18:38:56 +01:00
convert_xlnet_original_tf_checkpoint_to_pytorch.py
Remove __future__ imports.
2019-12-22 17:47:54 +01:00
file_utils.py
fix tqdm logging level
2019-12-23 16:47:24 +01:00
hf_api.py
Switch from comments to annotations for types.
2019-12-22 18:56:01 +01:00
modelcard.py
Use built-in open().
2019-12-22 18:38:56 +01:00
modeling_albert.py
Move source code inside a src subdirectory.
2019-12-22 14:15:13 +01:00
modeling_auto.py
Remove __future__ imports.
2019-12-22 17:47:54 +01:00
modeling_bert.py
Remove sys.version_info[0] == 2 or 3.
2019-12-22 18:38:42 +01:00
modeling_camembert.py
Remove __future__ imports.
2019-12-22 17:47:54 +01:00
modeling_ctrl.py
Remove __future__ imports.
2019-12-22 17:47:54 +01:00
modeling_distilbert.py
Remove __future__ imports.
2019-12-22 17:47:54 +01:00
modeling_encoder_decoder.py
Remove __future__ imports.
2019-12-22 17:47:54 +01:00
modeling_gpt2.py
Remove __future__ imports.
2019-12-22 17:47:54 +01:00
modeling_mmbt.py
Remove __future__ imports.
2019-12-22 17:47:54 +01:00
modeling_openai.py
Use built-in open().
2019-12-22 18:38:56 +01:00
modeling_roberta.py
Remove __future__ imports.
2019-12-22 17:47:54 +01:00
modeling_t5.py
Remove __future__ imports.
2019-12-22 17:47:54 +01:00
modeling_tf_albert.py
Remove sys.version_info[0] == 2 or 3.
2019-12-22 18:38:42 +01:00
modeling_tf_auto.py
Remove __future__ imports.
2019-12-22 17:47:54 +01:00
modeling_tf_bert.py
Remove sys.version_info[0] == 2 or 3.
2019-12-22 18:38:42 +01:00
modeling_tf_ctrl.py
Remove __future__ imports.
2019-12-22 17:47:54 +01:00
modeling_tf_distilbert.py
Remove __future__ imports.
2019-12-22 17:47:54 +01:00
modeling_tf_gpt2.py
Remove __future__ imports.
2019-12-22 17:47:54 +01:00
modeling_tf_openai.py
Remove __future__ imports.
2019-12-22 17:47:54 +01:00
modeling_tf_pytorch_utils.py
Simplify re-raising exceptions.
2019-12-23 21:20:54 +01:00
modeling_tf_roberta.py
Remove __future__ imports.
2019-12-22 17:47:54 +01:00
modeling_tf_t5.py
Remove __future__ imports.
2019-12-22 17:47:54 +01:00
modeling_tf_transfo_xl_utilities.py
Move source code inside a src subdirectory.
2019-12-22 14:15:13 +01:00
modeling_tf_transfo_xl.py
Remove __future__ imports.
2019-12-22 17:47:54 +01:00
modeling_tf_utils.py
Remove __future__ imports.
2019-12-22 17:47:54 +01:00
modeling_tf_xlm.py
Remove __future__ imports.
2019-12-22 17:47:54 +01:00
modeling_tf_xlnet.py
Remove sys.version_info[0] == 2 or 3.
2019-12-22 18:38:42 +01:00
modeling_transfo_xl_utilities.py
Move source code inside a src subdirectory.
2019-12-22 14:15:13 +01:00
modeling_transfo_xl.py
Remove __future__ imports.
2019-12-22 17:47:54 +01:00
modeling_utils.py
Simplify re-raising exceptions.
2019-12-23 21:20:54 +01:00
modeling_xlm_roberta.py
Remove __future__ imports.
2019-12-22 17:47:54 +01:00
modeling_xlm.py
Remove __future__ imports.
2019-12-22 17:47:54 +01:00
modeling_xlnet.py
Remove sys.version_info[0] == 2 or 3.
2019-12-22 18:38:42 +01:00
optimization_tf.py
Remove __future__ imports.
2019-12-22 17:47:54 +01:00
optimization.py
Move source code inside a src subdirectory.
2019-12-22 14:15:13 +01:00
pipelines.py
Improve exception type.
2019-12-23 21:27:38 +01:00
tokenization_albert.py
Reraise ImportError when sentencepiece isn't installed.
2019-12-23 21:27:42 +01:00
tokenization_auto.py
Remove __future__ imports.
2019-12-22 17:47:54 +01:00
tokenization_bert_japanese.py
Remove six.
2019-12-22 17:56:09 +01:00
tokenization_bert.py
FastPreTrainedTokenizer => PreTrainedTokenizerFast
2019-12-26 14:39:39 -05:00
tokenization_camembert.py
Remove six.
2019-12-22 17:56:09 +01:00
tokenization_ctrl.py
Use built-in open().
2019-12-22 18:38:56 +01:00
tokenization_distilbert.py
Remove __future__ imports.
2019-12-22 17:47:54 +01:00
tokenization_gpt2.py
FastPreTrainedTokenizer => PreTrainedTokenizerFast
2019-12-26 14:39:39 -05:00
tokenization_openai.py
Use built-in open().
2019-12-22 18:38:56 +01:00
tokenization_roberta.py
Remove import fallbacks.
2019-12-22 18:38:56 +01:00
tokenization_t5.py
Reraise ImportError when sentencepiece isn't installed.
2019-12-23 21:27:42 +01:00
tokenization_transfo_xl.py
Use the same pattern as everywhere else.
2019-12-23 21:30:04 +01:00
tokenization_utils.py
Handle unk_token
2019-12-26 14:42:55 -05:00
tokenization_xlm_roberta.py
Remove six.
2019-12-22 17:56:09 +01:00
tokenization_xlm.py
Simplify re-raising exceptions.
2019-12-23 21:20:54 +01:00
tokenization_xlnet.py
Reraise ImportError when sentencepiece isn't installed.
2019-12-23 21:27:42 +01:00