SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity

[s2s finetune_trainer] add instructions for distributed training (#8884)

Browse Source
This commit is contained in:
Stas Bekman
2020-12-03 16:05:55 -08:00
committed by GitHub
parent aa60b230ec
commit 4c3d98dddc
1 changed files with 5 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
examples/seq2seq/README.md
View File

@@ -213,6 +213,11 @@ To see all the possible command line options, run:
python finetune_trainer.py --help python finetune_trainer.py --help
``` ```
For multi-gpu training use `torch.distributed.launch`, e.g. with 2 gpus:
```bash
python -m torch.distributed.launch --nproc_per_node=2 finetune_trainer.py ...
```
**At the moment, `Seq2SeqTrainer` does not support *with teacher* distillation.** **At the moment, `Seq2SeqTrainer` does not support *with teacher* distillation.**
All `Seq2SeqTrainer`-based fine-tuning scripts are included in the `builtin_trainer` directory. All `Seq2SeqTrainer`-based fine-tuning scripts are included in the `builtin_trainer` directory.