SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity

[Deepspeed] various fixes (#12058)

Browse Source 
* replace deprecated config

* sub_group_size was too big

* complete deprecation removal
This commit is contained in:
Stas Bekman
2021-06-08 08:36:15 -07:00
committed by GitHub
parent fd6902838a
commit 32290d87f6
5 changed files with 54 additions and 39 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
tests/deepspeed/ds_config_zero3.json
View File

@@ -39,7 +39,7 @@
},
"overlap_comm": true,
"contiguous_gradients": true,
"sub_group_size": 1e14,
"sub_group_size": 1e9,
"reduce_bucket_size": "auto",
"stage3_prefetch_bucket_size": "auto",
"stage3_param_persistence_threshold": "auto",