SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity

🚨 Remove warning of deprecation (#4477)

Browse Source 
Remove warning of deprecated overload of addcdiv_

Fix #4451
This commit is contained in:
Cola
2020-05-21 05:48:29 +09:00
committed by GitHub
parent fa2fbed3e5
commit eacea530c1
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
src/transformers/optimization.py
View File

@@ -162,7 +162,7 @@ class AdamW(Optimizer):
bias_correction2 = 1.0 - beta2 ** state["step"] bias_correction2 = 1.0 - beta2 ** state["step"]
step_size = step_size * math.sqrt(bias_correction2) / bias_correction1 step_size = step_size * math.sqrt(bias_correction2) / bias_correction1
p.data.addcdiv_(-step_size, exp_avg, denom) p.data.addcdiv_(exp_avg, denom, value=-step_size)
# Just adding the square of the weights to the loss function is *not* # Just adding the square of the weights to the loss function is *not*
# the correct way of using L2 regularization/weight decay with Adam, # the correct way of using L2 regularization/weight decay with Adam,