Wing Lian 62aecd85ff schedulefree optimizers (#30079) ...

* schedulefree optimizers

* fix train instead of eval for optimizer

* fixes and update docs

* chore: lint

* add tests and drop overly-verbose _32bit suffix

* chore: lint

* fix for docs

* fix code review issues

* use duck-typing to avoid per-optimizer patches

* fixup style

* fixup style

* warn if incorrect accelerate version with schedule free

Co-authored-by: Aman Gupta Karmani <aman@tmm1.net>

---------

Co-authored-by: Aman Karmani <aman@tmm1.net>

2024-09-09 09:51:39 +02:00

..

__init__.py

[Test refactor 1/5] Per-folder tests reorganization (#15725)

2022-02-23 15:46:28 -05:00

test_data_collator.py

Enhancing SFT Training Efficiency Using Packing and FlashAttention2 with Position IDs (#31629)

2024-07-23 15:56:41 +02:00

test_trainer_callback.py

Modify warnings in a with block to avoid flaky tests (#31893)

2024-07-10 17:56:12 +02:00

test_trainer_distributed.py

CI: update to ROCm 6.0.2 and test MI300 (#30266)

2024-05-13 18:14:36 +02:00

test_trainer_seq2seq.py

Pass datasets trust_remote_code (#31406)

2024-06-17 17:29:13 +01:00

test_trainer_tpu.py

[Test refactor 1/5] Per-folder tests reorganization (#15725)

2022-02-23 15:46:28 -05:00

test_trainer_utils.py

Add strategy to store results in evaluation loop (#30267)

2024-04-17 12:42:27 +01:00

test_trainer.py

schedulefree optimizers (#30079)

2024-09-09 09:51:39 +02:00