Files
HuggingFace_transformer/tests
Nikolai Yakovenko 971d1802d0 Add AdaFactor optimizer from fairseq (#6722)
* AdaFactor optimizer ported from fairseq. Tested for T5 finetuning and MLM -- reduced memory consumption compared to ADAM.

* update PR fixes, add basic test

* bug -- incorrect params in test

* bugfix -- import Adafactor into test

* bugfix -- removed accidental T5 include

* resetting T5 to master

* bugfix -- include Adafactor in __init__

* longer loop for adafactor test

* remove double error class declare

* lint

* black

* isort

* Update src/transformers/optimization.py

Co-authored-by: Sam Shleifer <sshleifer@gmail.com>

* single docstring

* Cleanup docstring

Co-authored-by: Nikolai Y <nikolai.yakovenko@point72.com>
Co-authored-by: Sam Shleifer <sshleifer@gmail.com>
2020-08-27 04:58:13 -04:00
..
2020-05-07 13:48:44 -04:00
2020-08-24 11:03:01 -04:00
2020-08-26 11:10:36 -04:00
2020-08-26 17:20:22 +02:00
2020-08-26 17:20:22 +02:00
2020-08-26 17:20:22 +02:00
2020-08-26 17:20:22 +02:00
2020-08-26 17:20:22 +02:00
2020-08-26 17:20:22 +02:00
2020-08-26 17:20:22 +02:00
2020-08-26 17:20:22 +02:00
2020-08-26 17:20:22 +02:00
2020-08-26 17:20:22 +02:00
2020-08-26 17:20:22 +02:00
2020-08-26 17:20:22 +02:00
2020-08-26 17:20:22 +02:00
2020-08-26 17:20:22 +02:00
2020-08-26 17:20:22 +02:00
2020-08-26 17:20:22 +02:00
2020-08-26 17:20:22 +02:00
2020-08-26 17:20:22 +02:00
2020-08-26 17:20:22 +02:00
2020-08-26 17:20:22 +02:00
2020-08-26 17:20:22 +02:00
2020-08-26 17:20:22 +02:00
2020-08-26 17:20:22 +02:00
2020-08-26 17:20:22 +02:00
2020-08-26 17:20:22 +02:00
2020-08-26 17:20:22 +02:00
2020-08-26 17:20:22 +02:00
2020-08-26 17:20:22 +02:00
2020-08-26 17:20:22 +02:00
2020-08-26 17:20:22 +02:00
2020-08-17 09:04:35 -04:00
2020-08-26 17:20:22 +02:00
2020-08-26 17:20:22 +02:00
2020-08-26 17:20:22 +02:00
2020-08-26 17:20:22 +02:00
2020-08-26 17:20:22 +02:00
2020-08-26 17:20:22 +02:00
2020-08-26 17:20:22 +02:00