SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity

[hotfix] Fix frozen pooler parameters in SWAG example.

Browse Source
This commit is contained in:
Meet Pragnesh Shah
2019-06-11 15:13:53 -07:00
committed by GitHub
parent 784c0ed89a
commit e02ce4dc79
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
examples/run_swag.py
View File

@@ -390,7 +390,7 @@ def main():
# hack to remove pooler, which is not used # hack to remove pooler, which is not used
# thus it produce None grad that break apex # thus it produce None grad that break apex
param_optimizer = [n for n in param_optimizer if 'pooler' not in n[0]] param_optimizer = [n for n in param_optimizer]
no_decay = ['bias', 'LayerNorm.bias', 'LayerNorm.weight'] no_decay = ['bias', 'LayerNorm.bias', 'LayerNorm.weight']
optimizer_grouped_parameters = [ optimizer_grouped_parameters = [