@@ -193,8 +193,8 @@ Pass your text to the tokenizer:
|
|||||||
|
|
||||||
The tokenizer returns a dictionary containing:
|
The tokenizer returns a dictionary containing:
|
||||||
|
|
||||||
* [input_ids](./glossary#input-ids): numerical representions of your tokens.
|
* [input_ids](./glossary#input-ids): numerical representations of your tokens.
|
||||||
* [atttention_mask](.glossary#attention-mask): indicates which tokens should be attended to.
|
* [attention_mask](.glossary#attention-mask): indicates which tokens should be attended to.
|
||||||
|
|
||||||
A tokenizer can also accept a list of inputs, and pad and truncate the text to return a batch with uniform length:
|
A tokenizer can also accept a list of inputs, and pad and truncate the text to return a batch with uniform length:
|
||||||
|
|
||||||
@@ -525,4 +525,4 @@ All models are a standard [`tf.keras.Model`](https://www.tensorflow.org/api_docs
|
|||||||
|
|
||||||
## What's next?
|
## What's next?
|
||||||
|
|
||||||
Now that you've completed the 🤗 Transformers quick tour, check out our guides and learn how to do more specific things like writing a custom model, fine-tuning a model for a task, and how to train a model with a script. If you're interested in learning more about 🤗 Transformers core concepts, grab a cup of coffee and take a look at our Conceptual Guides!
|
Now that you've completed the 🤗 Transformers quick tour, check out our guides and learn how to do more specific things like writing a custom model, fine-tuning a model for a task, and how to train a model with a script. If you're interested in learning more about 🤗 Transformers core concepts, grab a cup of coffee and take a look at our Conceptual Guides!
|
||||||
|
|||||||
Reference in New Issue
Block a user