[trainer] --model_parallel hasn't been implemented for most models (#9347)

* --model_parallel hasn't been implemented for most models

* make the help clear as well

* implement is_parallelizable; use it

* oops

* remove property
This commit is contained in:
Stas Bekman
2021-01-05 01:01:30 -08:00
committed by GitHub
parent 4225740a7b
commit 748006c0b3
5 changed files with 12 additions and 2 deletions

View File

@@ -207,8 +207,8 @@ class TrainingArguments:
:obj:`"eval_loss"`.
- :obj:`False` if :obj:`metric_for_best_model` is not set, or set to :obj:`"loss"` or :obj:`"eval_loss"`.
model_parallel (:obj:`bool`, `optional`, defaults to :obj:`False`):
If there is more than one device, whether to use model parallelism to distribute the model's modules across
devices or not.
If the model supports model parallelism and there is more than one device, whether to use model parallelism
to distribute the model's modules across devices or not.
ignore_skip_data (:obj:`bool`, `optional`, defaults to :obj:`False`):
When resuming training, whether or not to skip the epochs and batches to get the data loading at the same
stage as in the previous training. If set to :obj:`True`, the training will begin faster (as that skipping