Sanchit Gandhi
a9701953ff
[whisper] static kv cache (#31166)
* make work with cache abstraction
* correct for static cache
* hacks for compile
* make fast
* fix
* fix pos ids
* generate
* fix sdpa
* fix sdpa cache pos
* fix fa2
* clean fa2
* integrate cache into generate
* make style
* copies
* more copies
* update eager
* update sdpa
* update fa2
* simplify
* use cache pos
* always compute cross-cache for debug
* avoid recompiles
Co-authored-by: Arthur Zucker <arthur@huggingface.co>
* fix fix
* fix fix fix
* more fix
* try encoder-decoder cache (too messy)
* revert encoder-decoder cache
* check cross-attn cache
* use enc-dec dataclass
* use richer enc-dec dataclass
* clean-up
* revert static cache changes
* small fixes
* revert to cpu flag
* fix copies
* add static slow test
* past k/v docstring
* more docstrings
* cache_position docstrings
* add to docs
* add enc-dec cache to docs
* make style
* fix after rebase
* fix beam
* style
* fix generation strategies
* fix most decoder-only tests
* style
* skip test
* more clean up
* small docstrings
* Apply suggestions from code review
Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com>
* add todo
* only crop self-attn
* check cache in mixin
* style
* fix re-compile after rebase
* move `is_updated` logic to enc-dec wrapper
* revert back
* revert cache back
* finalise design
* fix
* fix fix
* style
* Update src/transformers/cache_utils.py
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
* deprecate
* updates
* final updates
* style
* style
---------
Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com>
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
2024-07-02 13:24:15 +01:00
..
2023-11-16 11:44:36 -08:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2024-05-16 10:56:11 +01:00
2024-03-20 15:41:03 +00:00
2023-11-03 10:57:03 -04:00
2023-12-09 05:38:14 +09:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2024-02-16 08:16:58 +01:00
2023-11-03 10:57:03 -04:00
2024-04-26 16:23:44 +01:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2023-12-06 10:38:21 -08:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2024-06-04 18:29:45 +02:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2024-01-29 15:46:32 +01:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2023-11-06 19:45:03 +00:00
2024-02-14 08:41:31 +01:00
2023-11-03 10:57:03 -04:00
2023-11-10 13:49:10 +00:00
2024-05-09 22:57:52 +02:00
2024-04-17 12:19:18 +02:00
2024-03-15 14:29:11 +01:00
2024-05-08 11:42:07 +01:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2024-05-27 14:57:43 +02:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2024-05-08 11:42:07 +01:00
2024-05-16 10:56:11 +01:00
2023-11-03 10:57:03 -04:00
2024-02-19 15:22:29 +01:00
2024-05-28 18:07:07 +01:00
2024-05-08 11:42:07 +01:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2023-11-23 17:44:08 +00:00
2024-02-16 08:16:58 +01:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2023-11-23 17:44:08 +00:00
2024-05-28 18:07:07 +01:00
2023-06-20 18:07:47 -04:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2024-02-16 08:16:58 +01:00
2024-05-28 18:07:07 +01:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2023-09-04 17:18:34 +01:00
2024-03-29 14:31:31 +00:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2024-03-12 10:16:21 +00:00
2024-06-27 17:36:19 +02:00
2024-05-20 10:06:57 +02:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2024-06-19 09:40:57 +02:00
2024-02-16 08:16:58 +01:00
2023-12-09 05:38:14 +09:00
2023-11-03 10:57:03 -04:00
2024-06-26 13:56:36 +01:00
2024-01-15 09:09:22 +01:00
2023-11-03 10:57:03 -04:00
2024-05-28 18:07:07 +01:00
2024-05-28 18:07:07 +01:00
2024-04-19 21:03:07 +02:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2024-04-22 18:30:38 +01:00
2023-11-03 10:57:03 -04:00
2024-05-21 19:38:02 +02:00
2024-05-13 15:59:46 +01:00
2023-11-03 10:57:03 -04:00
2024-02-02 08:45:00 +01:00
2024-06-25 15:45:39 +05:00
2024-06-25 15:45:39 +05:00
2024-04-18 11:04:02 +02:00
2024-05-14 16:32:01 +02:00
2024-05-28 18:07:07 +01:00
2023-10-30 21:42:19 +01:00
2023-11-03 10:57:03 -04:00
2024-02-12 10:48:31 -08:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2024-02-12 10:48:31 -08:00
2023-12-20 14:25:07 +05:30
2024-06-27 10:32:51 -07:00
2024-05-20 10:06:57 +02:00
2024-05-20 13:45:56 +05:00
2024-06-26 21:52:28 +05:00
2024-04-11 10:38:24 +02:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2024-04-18 10:27:58 +02:00
2023-11-28 13:19:50 +00:00
2024-03-11 09:46:24 +01:00
2023-11-03 10:57:03 -04:00
2024-02-02 08:45:00 +01:00
2024-05-31 16:56:17 +01:00
2024-05-31 16:56:17 +01:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2024-05-28 18:07:07 +01:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2024-02-08 14:13:35 -08:00
2024-05-23 17:48:49 +01:00
2024-05-20 10:06:57 +02:00
2023-11-03 10:57:03 -04:00
2024-02-02 08:45:00 +01:00
2023-11-06 19:45:03 +00:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2024-02-01 03:53:49 +01:00
2024-03-18 13:06:12 +00:00
2024-02-12 10:48:31 -08:00
2023-11-03 10:57:03 -04:00
2024-05-28 18:07:07 +01:00
2024-05-28 18:07:07 +01:00
2023-11-06 19:45:03 +00:00
2024-04-18 10:27:58 +02:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2024-04-17 17:59:07 +02:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2023-12-09 05:38:14 +09:00
2024-05-14 08:36:11 +02:00
2023-12-11 18:03:42 +00:00
2024-05-28 13:22:06 +02:00
2024-02-19 15:22:29 +01:00
2024-02-19 15:22:29 +01:00
2024-02-08 14:13:35 -08:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2024-05-20 10:06:57 +02:00
2024-05-02 15:30:21 +02:00
2024-04-15 14:10:59 +02:00
2023-11-03 10:57:03 -04:00
2024-02-26 08:42:24 -08:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2024-02-12 10:48:31 -08:00
2023-11-03 10:57:03 -04:00
2024-03-13 19:05:20 +00:00
2024-02-08 14:13:35 -08:00
2024-05-28 18:07:07 +01:00
2024-05-20 10:06:57 +02:00
2024-05-20 10:06:57 +02:00
2023-11-03 10:57:03 -04:00
2024-05-28 18:07:07 +01:00
2024-04-15 18:30:59 +02:00
2024-02-02 08:45:00 +01:00
2023-11-06 19:45:03 +00:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2023-07-13 11:46:54 -04:00
2023-11-06 19:45:03 +00:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2024-06-26 14:42:20 +01:00
2024-02-02 08:45:00 +01:00
2024-05-31 12:34:29 +02:00
2023-12-14 15:14:13 +00:00
2023-12-14 15:14:13 +00:00
2024-06-10 12:35:10 +01:00
2024-04-26 19:40:12 +01:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2024-06-19 11:11:44 +01:00
2024-05-28 18:07:07 +01:00
2023-11-03 10:57:03 -04:00
2024-02-16 08:16:58 +01:00
2023-08-03 14:12:07 +01:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2024-05-20 10:06:57 +02:00
2024-05-20 10:06:57 +02:00
2024-05-28 13:22:06 +02:00
2024-04-19 18:31:43 +01:00
2023-06-20 18:07:47 -04:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2023-11-06 19:45:03 +00:00
2024-04-15 14:10:59 +02:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2023-11-06 19:45:03 +00:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2024-02-16 08:16:58 +01:00
2023-11-03 10:57:03 -04:00
2024-05-28 18:07:07 +01:00
2023-11-23 17:02:16 +00:00
2024-04-10 16:02:50 +02:00
2023-11-03 10:57:03 -04:00
2024-02-02 08:45:00 +01:00
2024-02-02 08:45:00 +01:00
2023-11-03 10:57:03 -04:00
2023-11-22 17:21:36 +01:00
2023-11-03 10:57:03 -04:00
2024-02-02 08:45:00 +01:00
2024-05-22 16:56:41 +05:00
2024-05-16 10:56:11 +01:00
2023-11-03 10:57:03 -04:00
2023-12-15 20:16:47 +01:00
2024-02-16 08:16:58 +01:00
2023-11-03 10:57:03 -04:00
2024-02-16 08:16:58 +01:00
2024-05-28 18:07:07 +01:00
2024-05-16 10:56:11 +01:00
2024-05-16 10:56:11 +01:00
2024-06-11 15:47:38 +01:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2024-01-18 13:37:34 +00:00
2024-06-05 11:56:11 +01:00
2024-06-05 11:56:11 +01:00
2024-02-02 08:45:00 +01:00
2024-07-02 13:24:15 +01:00
2024-05-28 18:07:07 +01:00
2023-11-03 10:57:03 -04:00
2024-05-28 18:07:07 +01:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2023-11-03 10:57:03 -04:00
2024-06-05 11:56:11 +01:00
2023-11-03 10:57:03 -04:00
2024-05-16 10:56:11 +01:00
2023-11-03 10:57:03 -04:00