SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity

[s2s]Use prepare_translation_batch for Marian finetuning (#6293)

Browse Source 
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
This commit is contained in:
Sam Shleifer
2020-08-06 14:58:38 -04:00
committed by GitHub
parent 2f2aa0c89c
commit 2804fff839
5 changed files with 22 additions and 12 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
examples/seq2seq/README.md
View File

@@ -63,7 +63,7 @@ Summarization Tips:
(It rarely makes sense to start from `bart-large` unless you are a researching finetuning methods).
**Update 2018-07-18**
Datasets: Seq2SeqDataset will be used for all models besides MBart, for which MBartDataset will be used.**
Datasets: `Seq2SeqDataset` should be used for all tokenizers without a `prepare_translation_batch` method. For those who do (like Marian, MBart), `TranslationDataset` should be used.**
A new dataset is needed to support multilingual tasks.