Trainer - deprecate tokenizer for processing_class (#32385)

* Trainer - deprecate tokenizer for processing_class

* Extend chage across Seq2Seq trainer and docs

* Add tests

* Update to FutureWarning and add deprecation version
This commit is contained in:
amyeroberts
2024-10-02 14:08:46 +01:00
committed by GitHub
parent e7c8af7f33
commit b7474f211d
99 changed files with 569 additions and 442 deletions

View File

@@ -298,7 +298,7 @@ Wav2Vec2 分词器仅训练了大写字符,因此您需要确保文本与分
... args=training_args,
... train_dataset=encoded_minds["train"],
... eval_dataset=encoded_minds["test"],
... tokenizer=processor,
... processing_class=processor,
... data_collator=data_collator,
... compute_metrics=compute_metrics,
... )
@@ -389,4 +389,4 @@ Wav2Vec2 分词器仅训练了大写字符,因此您需要确保文本与分
['I WOUL LIKE O SET UP JOINT ACOUNT WTH Y PARTNER']
```
</pt>
</frameworkcontent>
</frameworkcontent>