Mark "code in the Hub" API as experimental (#15624)

This commit is contained in:
Sylvain Gugger
2022-02-11 09:55:31 -05:00
committed by GitHub
parent 45c7b5b1c7
commit 6cf06d198c
6 changed files with 36 additions and 0 deletions

View File

@@ -3417,6 +3417,12 @@ class PreTrainedTokenizerBase(SpecialTokensMixin, PushToHubMixin):
Register this class with a given auto class. This should only be used for custom tokenizers as the ones in the
library are already mapped with `AutoTokenizer`.
<Tip warning={true}>
This API is experimental and may have some slight breaking changes in the next releases.
</Tip>
Args:
auto_class (`str` or `type`, *optional*, defaults to `"AutoTokenizer"`):
The auto class to register this new tokenizer with.