Fix rendering issue in quicktour (#20708)
* Fix rendering issue in quicktour * Separate in two blocks
This commit is contained in:
@@ -440,13 +440,16 @@ Depending on your task, you'll typically pass the following parameters to [`Trai
|
|||||||
>>> dataset = load_dataset("rotten_tomatoes") # doctest: +IGNORE_RESULT
|
>>> dataset = load_dataset("rotten_tomatoes") # doctest: +IGNORE_RESULT
|
||||||
```
|
```
|
||||||
|
|
||||||
5. Create a function to tokenize the dataset, and apply it over the entire dataset with [`~datasets.Dataset.map`]:
|
5. Create a function to tokenize the dataset:
|
||||||
|
|
||||||
```py
|
```py
|
||||||
>>> def tokenize_dataset(dataset):
|
>>> def tokenize_dataset(dataset):
|
||||||
... return tokenizer(dataset["text"])
|
... return tokenizer(dataset["text"])
|
||||||
|
```
|
||||||
|
|
||||||
|
Then apply it over the entire dataset with [`~datasets.Dataset.map`]:
|
||||||
|
|
||||||
|
```py
|
||||||
>>> dataset = dataset.map(tokenize_dataset, batched=True)
|
>>> dataset = dataset.map(tokenize_dataset, batched=True)
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user