SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity

A few fixes in the documentation (#10033)

Browse Source
This commit is contained in:
Sylvain Gugger
2021-02-08 05:02:01 -05:00
committed by GitHub
parent 04fd783cc5
commit 45aaf5f7ab
6 changed files with 12 additions and 17 deletions
Download Patch File Download Diff File Expand all files Collapse all files

12
docs/source/main_classes/tokenizer.rst
View File

@@ -54,9 +54,9 @@ PreTrainedTokenizer
.. autoclass:: transformers.PreTrainedTokenizer
:special-members: __call__
:members:
.. automethod:: encode
:members: batch_decode, convert_ids_to_tokens, convert_tokens_to_ids, convert_tokens_to_string, decode, encode,
get_added_vocab, get_special_tokens_mask, num_special_tokens_to_add, prepare_for_tokenization, tokenize,
vocab_size
PreTrainedTokenizerFast
@@ -64,9 +64,9 @@ PreTrainedTokenizerFast
.. autoclass:: transformers.PreTrainedTokenizerFast
:special-members: __call__
:members:
.. automethod:: encode
:members: batch_decode, convert_ids_to_tokens, convert_tokens_to_ids, convert_tokens_to_string, decode, encode,
get_added_vocab, get_special_tokens_mask, num_special_tokens_to_add,
set_truncation_and_padding,tokenize, vocab_size
BatchEncoding