Sylvain Gugger b70f441b72 Smp grad accum (#10488) ...

* Fix gradient accumulation for SM Model Parallelism

* Style and divide loss by grad accum steps

2021-03-03 12:13:29 -05:00

..

transformers

Smp grad accum (#10488)

2021-03-03 12:13:29 -05:00