Files
HuggingFace_transformer/docs/source/en/model_doc
Eduardo Pacheco 22d159ddf9 Adding Flash Attention 2 Support for GPT2 (#29226)
* First commit to add flash attention 2 for GPT-2

* more improvements

* Make GPT2 pass tests and fixed Decison Transformers copies

* Fixed missing arg

* fix copies

* Added expected speedup

* Update src/transformers/models/gpt2/modeling_gpt2.py

Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>

* Update src/transformers/models/gpt2/modeling_gpt2.py

Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>

* Update src/transformers/models/gpt2/modeling_gpt2.py

Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>

* Added test

* Fixed attn attribute

* Update docs/source/en/model_doc/gpt2.md

Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>

* Update docs/source/en/model_doc/gpt2.md

Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>

* Update Decision transformer attentions

* More updates

* Passing tests

* Fix copies

* Fix copies part 2

* Decision transformer updates

* Update src/transformers/models/gpt2/modeling_gpt2.py

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

* Fix copies

* Decision transformer not supporting flash attn

* Addressed comments

* Addressed comments

* Addressed comments

---------

Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
2024-03-28 09:31:24 +00:00
..
2024-03-11 17:26:38 +00:00
2023-11-06 19:45:03 +00:00
2023-11-10 13:49:10 +00:00
2024-03-15 14:29:11 +01:00
2023-11-23 17:44:08 +00:00
2023-11-23 17:44:08 +00:00
2024-03-12 10:16:21 +00:00
2024-02-21 14:21:28 +01:00
2023-10-30 21:42:19 +01:00
2023-12-20 14:25:07 +05:30
2023-12-11 10:22:26 +01:00
2023-11-13 14:20:54 +01:00
2024-02-19 15:22:29 +01:00
2024-02-19 15:22:29 +01:00
2024-01-22 17:15:07 +00:00
2024-03-13 19:05:20 +00:00
2024-03-27 02:11:55 +01:00
2024-02-23 10:43:31 +01:00
2023-11-06 19:45:03 +00:00
2023-07-13 11:46:54 -04:00
2024-02-19 15:22:29 +01:00
2024-02-19 15:22:29 +01:00
2024-02-14 07:15:18 +01:00
2023-11-06 19:45:03 +00:00
2023-11-23 17:02:16 +00:00
2024-03-04 18:49:02 +01:00
2023-12-15 20:16:47 +01:00
2024-02-19 15:22:29 +01:00