SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity

[Fix doc example] Fix first example for the custom_datasets tutorial (#16087)

Browse Source 
* Fix inconsistent example variable naming

- Example code for a sequence classification in Tensorflow had spelling mistakes and incorrect and inconsistent naming
- Changed variable naming to be consistent with the two other TF examples

* Fix incorrect incorrect training examples
This commit is contained in:
Markus Sagen
2022-03-15 13:17:51 +01:00
committed by GitHub
parent 8bfd2fb8f0
commit bcaf566038
1 changed files with 4 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
docs/source/custom_datasets.mdx
View File

@@ -163,14 +163,14 @@ Next, convert your datasets to the `tf.data.Dataset` format with `to_tf_dataset`
`columns` argument: `columns` argument:
```python ```python
tf_train_dataset = tokenized_imdb["train"].to_tf_dataset( tf_train_set = tokenized_imdb["train"].to_tf_dataset(
columns=["attention_mask", "input_ids", "label"], columns=["attention_mask", "input_ids", "label"],
shuffle=True, shuffle=True,
batch_size=16, batch_size=16,
collate_fn=data_collator, collate_fn=data_collator,
) )
tf_validation_dataset = tokenized_imdb["train"].to_tf_dataset( tf_validation_set = tokenized_imdb["test"].to_tf_dataset(
columns=["attention_mask", "input_ids", "label"], columns=["attention_mask", "input_ids", "label"],
shuffle=False, shuffle=False,
batch_size=16, batch_size=16,
@@ -185,9 +185,9 @@ from transformers import create_optimizer
import tensorflow as tf import tensorflow as tf
batch_size = 16 batch_size = 16
num_epochs = 5 num_train_epochs = 5
batches_per_epoch = len(tokenized_imdb["train"]) // batch_size batches_per_epoch = len(tokenized_imdb["train"]) // batch_size
total_train_steps = int(batches_per_epoch * num_epochs) total_train_steps = int(batches_per_epoch * num_train_epochs)
optimizer, schedule = create_optimizer(init_lr=2e-5, num_warmup_steps=0, num_train_steps=total_train_steps) optimizer, schedule = create_optimizer(init_lr=2e-5, num_warmup_steps=0, num_train_steps=total_train_steps)
``` ```