Mark "code in the Hub" API as experimental (#15624)
This commit is contained in:
@@ -3417,6 +3417,12 @@ class PreTrainedTokenizerBase(SpecialTokensMixin, PushToHubMixin):
|
||||
Register this class with a given auto class. This should only be used for custom tokenizers as the ones in the
|
||||
library are already mapped with `AutoTokenizer`.
|
||||
|
||||
<Tip warning={true}>
|
||||
|
||||
This API is experimental and may have some slight breaking changes in the next releases.
|
||||
|
||||
</Tip>
|
||||
|
||||
Args:
|
||||
auto_class (`str` or `type`, *optional*, defaults to `"AutoTokenizer"`):
|
||||
The auto class to register this new tokenizer with.
|
||||
|
||||
Reference in New Issue
Block a user