[PretrainedTokenizer] add some of the most important functions to the doc (#27313)
This commit is contained in:
@@ -55,6 +55,8 @@ to a given token).
|
|||||||
|
|
||||||
[[autodoc]] PreTrainedTokenizer
|
[[autodoc]] PreTrainedTokenizer
|
||||||
- __call__
|
- __call__
|
||||||
|
- add_tokens
|
||||||
|
- add_special_tokens
|
||||||
- apply_chat_template
|
- apply_chat_template
|
||||||
- batch_decode
|
- batch_decode
|
||||||
- decode
|
- decode
|
||||||
@@ -69,6 +71,8 @@ loaded very simply into 🤗 transformers. Take a look at the [Using tokenizers
|
|||||||
|
|
||||||
[[autodoc]] PreTrainedTokenizerFast
|
[[autodoc]] PreTrainedTokenizerFast
|
||||||
- __call__
|
- __call__
|
||||||
|
- add_tokens
|
||||||
|
- add_special_tokens
|
||||||
- apply_chat_template
|
- apply_chat_template
|
||||||
- batch_decode
|
- batch_decode
|
||||||
- decode
|
- decode
|
||||||
|
|||||||
Reference in New Issue
Block a user