SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity

[Trainer] add disclaimer that full_determinism is slow (#22368)

Browse Source
This commit is contained in:
Stas Bekman
2023-03-24 12:46:41 -07:00
committed by GitHub
parent a0cbbba31f
commit 500fce073b
1 changed files with 2 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
src/transformers/training_args.py
View File

@@ -538,7 +538,7 @@ class TrainingArguments:
CUDA Out-of-Memory errors. Requires accelerate to be installed (`pip install accelerate`) CUDA Out-of-Memory errors. Requires accelerate to be installed (`pip install accelerate`)
full_determinism (`bool`, *optional*, defaults to `False`) full_determinism (`bool`, *optional*, defaults to `False`)
If `True`, [`enable_full_determinism`] is called instead of [`set_seed`] to ensure reproducible results in If `True`, [`enable_full_determinism`] is called instead of [`set_seed`] to ensure reproducible results in
distributed training distributed training. Important: this will negatively impact the performance, so only use it for debugging.
torchdynamo (`str`, *optional*): torchdynamo (`str`, *optional*):
If set, the backend compiler for TorchDynamo. Possible choices are `"eager"`, `"aot_eager"`, `"inductor"`, If set, the backend compiler for TorchDynamo. Possible choices are `"eager"`, `"aot_eager"`, `"inductor"`,
`"nvfuser"`, `"aot_nvfuser"`, `"aot_cudagraphs"`, `"ofi"`, `"fx2trt"`, `"onnxrt"` and `"ipex"`. `"nvfuser"`, `"aot_nvfuser"`, `"aot_cudagraphs"`, `"ofi"`, `"fx2trt"`, `"onnxrt"` and `"ipex"`.
@@ -1055,7 +1055,7 @@ class TrainingArguments:
metadata={ metadata={
"help": ( "help": (
"Whether to call enable_full_determinism instead of set_seed for reproducibility in distributed" "Whether to call enable_full_determinism instead of set_seed for reproducibility in distributed"
" training" " training. Important: this will negatively impact the performance, so only use it for debugging."
) )
}, },
) )