SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity

Document adam betas TrainingArguments (#8688)

Browse Source
This commit is contained in:
Sylvain Gugger
2020-11-20 09:27:25 -05:00
committed by GitHub
parent 94caaa93c2
commit 63e91f5fde
1 changed files with 4 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
src/transformers/training_args.py
View File

@@ -89,6 +89,10 @@ class TrainingArguments:
The initial learning rate for Adam. The initial learning rate for Adam.
weight_decay (:obj:`float`, `optional`, defaults to 0): weight_decay (:obj:`float`, `optional`, defaults to 0):
The weight decay to apply (if not zero). The weight decay to apply (if not zero).
adam_beta1 (:obj:`float`, `optional`, defaults to 0.9):
The beta1 for the Adam optimizer.
adam_beta2 (:obj:`float`, `optional`, defaults to 0.999):
The beta2 for the Adam optimizer.
adam_epsilon (:obj:`float`, `optional`, defaults to 1e-8): adam_epsilon (:obj:`float`, `optional`, defaults to 1e-8):
Epsilon for the Adam optimizer. Epsilon for the Adam optimizer.
max_grad_norm (:obj:`float`, `optional`, defaults to 1.0): max_grad_norm (:obj:`float`, `optional`, defaults to 1.0):