Younes Belkada
368a58e61c
[core ] Integrate Flash attention 2 in most used models (#25598)
* v1
* oops
* working v1
* fixup
* add some TODOs
* fixup
* padding support + try with module replacement
* nit
* alternative design
* oops
* add `use_cache` support for llama
* v1 falcon
* nit
* a bit of refactor
* nit
* nits nits
* add v1 padding support falcon (even though it seemed to work before)
* nit
* falcon works
* fixup
* v1 tests
* nit
* fix generation llama flash
* update tests
* fix tests + nits
* fix copies
* fix nit
* test- padding mask
* stype
* add more mem efficient support
* Update src/transformers/modeling_utils.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
* fixup
* nit
* fixup
* remove it from config when saving
* fixup
* revert docstring
* add more checks
* use values
* oops
* new version
* fixup
* add same trick for falcon
* nit
* add another test
* change tests
* fix issues with GC and also falcon
* fixup
* oops
* Update src/transformers/models/falcon/modeling_falcon.py
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
* add init_rope
* updates
* fix copies
* fixup
* fixup
* more clarification
* fixup
* right padding tests
* add docs
* add FA in docker image
* more clarifications
* add some figures
* add todo
* rectify comment
* Change to FA2
* Update docs/source/en/perf_infer_gpu_one.md
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
* split in two lines
* change test name
* add more tests
* some clean up
* remove `rearrange` deps
* add more docs
* revert changes on dockerfile
* Revert "revert changes on dockerfile"
This reverts commit 8d72a66b4b9b771abc3f15a9b9506b4246d62d8e.
* revert changes on dockerfile
* Apply suggestions from code review
Co-authored-by: Lysandre Debut <hi@lysand.re>
* address some comments
* docs
* use inheritance
* Update src/transformers/testing_utils.py
Co-authored-by: Lysandre Debut <hi@lysand.re>
* fixup
* Apply suggestions from code review
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
* Update src/transformers/modeling_utils.py
* final comments
* clean up
* style
* add cast + warning for PEFT models
* fixup
---------
Co-authored-by: Felix Marty <9808326+fxmarty@users.noreply.github.com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
Co-authored-by: Lysandre Debut <hi@lysand.re>
2023-09-22 17:42:10 +02:00
..
2023-08-02 20:22:36 +02:00
2023-08-02 20:22:36 +02:00
2023-08-19 11:15:38 +02:00
2023-08-02 20:22:36 +02:00
2023-06-06 14:31:14 -04:00
2023-06-26 18:36:47 +02:00
2023-08-09 18:28:02 +02:00
2023-09-18 20:28:36 +02:00
2023-02-06 18:10:56 -05:00
2023-02-06 18:10:56 -05:00
2023-08-11 11:30:18 +01:00
2023-08-02 20:22:36 +02:00
2023-08-02 20:22:36 +02:00
2023-02-06 18:10:56 -05:00
2023-02-06 18:10:56 -05:00
2023-06-22 16:11:27 +02:00
2023-08-02 20:22:36 +02:00
2023-08-02 20:22:36 +02:00
2023-06-29 10:17:36 +01:00
2023-09-14 15:10:34 +01:00
2023-08-02 20:22:36 +02:00
2023-08-16 17:45:02 +01:00
2023-08-02 20:22:36 +02:00
2023-09-18 20:28:36 +02:00
2023-08-11 11:30:18 +01:00
2023-09-20 10:22:07 +02:00
2023-09-18 20:28:36 +02:00
2023-09-18 20:28:36 +02:00
2023-09-18 20:28:36 +02:00
2023-08-16 17:45:02 +01:00
2023-08-02 20:22:36 +02:00
2023-08-25 10:58:14 +02:00
2023-08-08 10:48:45 +02:00
2023-09-20 17:19:30 +02:00
2023-09-18 20:28:36 +02:00
2023-08-11 11:30:18 +01:00
2023-08-02 20:22:36 +02:00
2023-08-11 11:30:18 +01:00
2023-05-16 23:35:11 +02:00
2023-03-22 20:02:24 +01:00
2023-08-02 20:22:36 +02:00
2023-09-18 13:52:43 +02:00
2023-07-24 10:08:47 -04:00
2023-09-05 10:12:25 +02:00
2023-08-02 20:22:36 +02:00
2023-09-18 20:28:36 +02:00
2023-02-28 19:40:57 +01:00
2023-08-11 11:30:18 +01:00
2023-08-11 11:30:18 +01:00
2023-08-11 11:30:18 +01:00
2023-09-05 11:19:56 +02:00
2023-06-29 10:17:36 +01:00
2023-08-29 11:05:27 +01:00
2023-08-02 20:22:36 +02:00
2023-06-29 10:17:36 +01:00
2023-08-11 11:30:18 +01:00
2023-08-02 20:22:36 +02:00
2023-08-11 11:30:18 +01:00
2023-08-11 11:30:18 +01:00
2023-08-14 17:08:47 +01:00
2023-08-02 20:22:36 +02:00
2023-07-24 10:08:47 -04:00
2023-06-06 18:30:51 +01:00
2023-08-02 20:22:36 +02:00
2023-08-02 20:22:36 +02:00
2023-08-08 10:48:45 +02:00
2023-09-13 17:00:52 +01:00
2023-08-02 20:22:36 +02:00
2023-08-16 17:45:02 +01:00
2023-08-02 20:22:36 +02:00
2023-05-18 17:29:04 +02:00
2023-09-21 14:46:05 +02:00
2023-06-16 15:40:49 +01:00
2023-08-02 20:22:36 +02:00
2023-08-16 17:45:02 +01:00
2023-09-18 20:28:36 +02:00
2023-08-02 20:22:36 +02:00
2023-08-02 20:22:36 +02:00
2023-08-02 20:22:36 +02:00
2023-08-02 20:22:36 +02:00
2023-09-18 20:28:36 +02:00
2023-08-02 20:22:36 +02:00
2023-09-18 20:28:36 +02:00
2023-07-24 10:08:47 -04:00
2023-08-02 20:22:36 +02:00
2023-02-06 18:10:56 -05:00
2023-09-05 10:12:25 +02:00
2023-08-02 20:22:36 +02:00
2023-09-14 19:27:40 -04:00
2023-08-16 17:45:02 +01:00
2023-06-22 16:11:27 +02:00
2023-08-31 17:01:27 +02:00
2023-08-18 12:40:40 +02:00
2023-08-02 20:22:36 +02:00
2023-09-14 15:10:34 +01:00
2023-09-18 20:28:36 +02:00
2023-09-14 15:10:34 +01:00
2023-06-30 16:30:33 +01:00
2023-08-11 11:30:18 +01:00
2023-06-22 16:11:27 +02:00
2023-09-22 17:42:10 +02:00
2023-08-07 17:45:41 +02:00
2023-08-02 20:22:36 +02:00
2023-09-18 20:28:36 +02:00
2023-06-16 15:40:49 +01:00
2023-09-18 20:28:36 +02:00
2023-09-12 21:53:31 +02:00
2023-09-18 20:28:36 +02:00
2023-08-11 11:30:18 +01:00
2023-08-16 18:11:24 +02:00
2023-08-02 20:22:36 +02:00
2022-07-29 08:09:09 -04:00
2023-09-05 21:50:14 +02:00
2023-08-02 20:22:36 +02:00
2022-05-03 14:42:02 +02:00
2023-08-11 11:30:18 +01:00
2023-09-18 20:28:36 +02:00
2023-08-19 11:15:38 +02:00
2023-08-11 11:30:18 +01:00
2023-08-11 11:30:18 +01:00
2023-08-11 11:30:18 +01:00
2023-07-24 10:08:47 -04:00
2023-08-02 20:22:36 +02:00
2023-08-21 11:11:21 +02:00
2023-08-02 20:22:36 +02:00
2023-05-24 13:52:52 +01:00
2023-09-15 10:00:36 +01:00
2023-08-02 20:22:36 +02:00
2023-06-29 10:17:36 +01:00
2023-08-02 20:22:36 +02:00
2023-04-04 14:53:06 +02:00
2023-08-17 17:21:56 +02:00
2023-08-02 20:22:36 +02:00
2023-08-11 11:30:18 +01:00
2023-08-02 20:22:36 +02:00
2023-08-18 12:40:40 +02:00
2023-09-18 20:28:36 +02:00
2023-09-18 20:28:36 +02:00
2023-08-02 20:22:36 +02:00
2023-09-18 20:28:36 +02:00
2023-09-12 11:33:27 +02:00
2023-02-06 18:10:56 -05:00
2023-08-16 17:45:02 +01:00
2023-08-02 20:22:36 +02:00
2023-08-11 11:30:18 +01:00
2023-08-29 18:00:40 +01:00
2023-08-02 20:22:36 +02:00
2023-08-11 11:30:18 +01:00
2023-08-02 20:22:36 +02:00
2023-06-16 15:40:49 +01:00
2023-08-02 20:22:36 +02:00
2023-05-18 11:04:51 +01:00
2023-06-29 10:17:36 +01:00
2023-08-02 20:22:36 +02:00
2023-06-29 10:17:36 +01:00
2023-09-18 20:28:36 +02:00
2023-08-08 10:48:45 +02:00
2023-08-02 20:22:36 +02:00
2023-08-02 20:22:36 +02:00
2023-08-02 20:22:36 +02:00
2023-06-22 16:11:27 +02:00
2023-08-11 11:30:18 +01:00
2023-09-05 10:12:25 +02:00
2023-09-05 10:12:25 +02:00
2023-02-06 18:10:56 -05:00
2023-09-18 20:28:36 +02:00
2023-08-02 20:22:36 +02:00
2023-09-05 10:12:25 +02:00
2023-08-02 20:22:36 +02:00
2023-08-02 20:22:36 +02:00
2023-07-25 07:56:04 -04:00
2023-06-29 10:17:36 +01:00
2023-08-16 17:45:02 +01:00
2023-06-29 10:17:36 +01:00
2023-08-11 13:16:01 +01:00
2023-09-21 19:00:03 +09:00
2023-07-25 07:56:04 -04:00
2023-09-14 15:10:34 +01:00
2023-06-22 16:11:27 +02:00
2023-08-02 20:22:36 +02:00
2023-09-18 23:47:54 +02:00
2023-08-02 20:22:36 +02:00
2023-08-02 20:22:36 +02:00
2023-08-16 17:45:02 +01:00
2023-08-02 20:22:36 +02:00
2023-09-05 10:12:25 +02:00
2023-09-05 10:12:25 +02:00
2023-07-25 07:56:04 -04:00
2023-08-16 17:45:02 +01:00
2023-08-11 11:30:18 +01:00
2023-06-29 10:17:36 +01:00
2023-07-05 13:44:30 +02:00
2023-08-02 20:22:36 +02:00
2023-08-11 11:30:18 +01:00
2023-08-16 17:02:57 +02:00
2023-08-02 20:22:36 +02:00
2023-08-02 20:22:36 +02:00
2023-09-04 11:35:39 +02:00
2023-09-19 10:56:10 -03:00
2023-09-18 20:28:36 +02:00
2023-08-16 17:45:02 +01:00
2023-09-18 20:28:36 +02:00
2023-09-05 18:26:06 +01:00
2022-05-03 14:42:02 +02:00
2023-04-17 12:41:55 +02:00
2023-09-05 10:12:25 +02:00
2023-09-15 15:46:31 +01:00
2023-08-02 20:22:36 +02:00
2023-08-18 12:40:40 +02:00
2023-08-02 20:22:36 +02:00
2023-03-06 09:15:44 +01:00
2023-05-24 13:52:52 +01:00
2023-08-02 20:22:36 +02:00
2023-09-18 20:28:36 +02:00
2023-08-02 20:22:36 +02:00
2023-08-11 11:30:18 +01:00
2023-08-02 20:22:36 +02:00
2022-05-03 14:42:02 +02:00