Nikolai Yakovenko
971d1802d0
Add AdaFactor optimizer from fairseq ( #6722 )
...
* AdaFactor optimizer ported from fairseq. Tested for T5 finetuning and MLM -- reduced memory consumption compared to ADAM.
* update PR fixes, add basic test
* bug -- incorrect params in test
* bugfix -- import Adafactor into test
* bugfix -- removed accidental T5 include
* resetting T5 to master
* bugfix -- include Adafactor in __init__
* longer loop for adafactor test
* remove double error class declare
* lint
* black
* isort
* Update src/transformers/optimization.py
Co-authored-by: Sam Shleifer <sshleifer@gmail.com >
* single docstring
* Cleanup docstring
Co-authored-by: Nikolai Y <nikolai.yakovenko@point72.com >
Co-authored-by: Sam Shleifer <sshleifer@gmail.com >
2020-08-27 04:58:13 -04:00
..
2020-05-07 13:48:44 -04:00
2020-01-06 15:11:12 +01:00
2020-07-31 04:34:46 -04:00
2020-08-24 11:03:01 -04:00
2020-08-24 11:03:01 -04:00
2020-08-24 11:03:01 -04:00
2020-07-01 10:31:17 -04:00
2020-03-05 17:16:57 -05:00
2020-08-20 11:13:50 -04:00
2020-07-01 10:31:17 -04:00
2020-08-24 11:03:01 -04:00
2020-07-31 04:44:23 -04:00
2020-08-26 11:10:36 -04:00
2020-01-06 15:11:12 +01:00
2020-08-26 17:20:22 +02:00
2020-08-24 11:03:01 -04:00
2020-08-26 17:20:22 +02:00
2020-08-26 17:20:22 +02:00
2020-08-26 17:20:22 +02:00
2020-08-26 17:20:22 +02:00
2020-08-26 17:20:22 +02:00
2020-08-26 17:20:22 +02:00
2020-08-26 17:20:22 +02:00
2020-08-26 17:20:22 +02:00
2020-08-26 17:20:22 +02:00
2020-08-26 17:20:22 +02:00
2020-08-26 17:20:22 +02:00
2020-08-26 17:20:22 +02:00
2020-08-24 11:03:01 -04:00
2020-08-24 11:03:01 -04:00
2020-08-26 17:20:22 +02:00
2020-08-26 17:20:22 +02:00
2020-08-25 14:06:28 -04:00
2020-08-26 17:20:22 +02:00
2020-08-26 17:20:22 +02:00
2020-08-26 17:20:22 +02:00
2020-08-24 11:03:01 -04:00
2020-08-24 11:03:01 -04:00
2020-08-24 11:03:01 -04:00
2020-08-26 17:20:22 +02:00
2020-08-26 23:16:06 +02:00
2020-08-26 17:20:22 +02:00
2020-08-26 17:20:22 +02:00
2020-08-26 17:20:22 +02:00
2020-08-26 17:20:22 +02:00
2020-08-26 17:20:22 +02:00
2020-08-26 14:55:41 -04:00
2020-08-24 11:03:01 -04:00
2020-08-26 17:20:22 +02:00
2020-08-26 17:20:22 +02:00
2020-08-26 17:20:22 +02:00
2020-08-26 17:20:22 +02:00
2020-08-24 11:03:01 -04:00
2020-08-26 17:20:22 +02:00
2020-08-26 17:20:22 +02:00
2020-08-26 17:20:22 +02:00
2020-08-24 11:03:01 -04:00
2020-08-26 17:20:22 +02:00
2020-08-26 17:20:22 +02:00
2020-08-17 09:04:35 -04:00
2020-08-24 11:03:01 -04:00
2020-08-27 04:58:13 -04:00
2020-08-26 17:20:22 +02:00
2020-06-15 17:12:51 -04:00
2020-07-01 10:31:17 -04:00
2020-08-17 12:00:23 +08:00
2020-08-06 18:52:28 +08:00
2020-08-26 17:20:22 +02:00
2020-06-15 17:12:51 -04:00
2020-07-01 10:31:17 -04:00
2020-07-07 16:35:12 +02:00
2020-08-26 17:20:22 +02:00
2020-06-15 17:12:51 -04:00
2020-08-11 15:57:07 -04:00
2020-08-26 17:20:22 +02:00
2020-06-15 17:12:51 -04:00
2020-08-11 14:31:23 -04:00
2020-08-26 17:20:22 +02:00
2020-07-01 10:31:17 -04:00
2020-08-26 17:20:22 +02:00
2020-08-24 11:03:01 -04:00
2020-08-12 08:00:56 -04:00
2020-07-01 10:31:17 -04:00
2020-07-01 10:31:17 -04:00
2020-07-01 10:31:17 -04:00
2020-08-20 11:13:50 -04:00
2020-08-26 17:20:22 +02:00