Sanchit Gandhi a9701953ff [whisper] static kv cache (#31166) ...

* make work with cache abstraction

* correct for static cache

* hacks for compile

* make fast

* fix

* fix pos ids

* generate

* fix sdpa

* fix sdpa cache pos

* fix fa2

* clean fa2

* integrate cache into generate

* make style

* copies

* more copies

* update eager

* update sdpa

* update fa2

* simplify

* use cache pos

* always compute cross-cache for debug

* avoid recompiles
Co-authored-by: Arthur Zucker <arthur@huggingface.co>

* fix fix

* fix fix fix

* more fix

* try encoder-decoder cache (too messy)

* revert encoder-decoder cache

* check cross-attn cache

* use enc-dec dataclass

* use richer enc-dec dataclass

* clean-up

* revert static cache changes

* small fixes

* revert to cpu flag

* fix copies

* add static slow test

* past k/v docstring

* more docstrings

* cache_position docstrings

* add to docs

* add enc-dec cache to docs

* make style

* fix after rebase

* fix beam

* style

* fix generation strategies

* fix most decoder-only tests

* style

* skip test

* more clean up

* small docstrings

* Apply suggestions from code review

Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com>

* add todo

* only crop self-attn

* check cache in mixin

* style

* fix re-compile after rebase

* move `is_updated` logic to enc-dec wrapper

* revert back

* revert cache back

* finalise design

* fix

* fix fix

* style

* Update src/transformers/cache_utils.py

Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>

* deprecate

* updates

* final updates

* style

* style

---------

Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com>
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>

2024-07-02 13:24:15 +01:00

..

agents

Agents: Improve python interpreter (#31409)

2024-06-18 11:55:36 +02:00

benchmark

[Test refactor 1/5] Per-folder tests reorganization (#15725)

2022-02-23 15:46:28 -05:00

bettertransformer

Fixed malapropism error (#26660)

2023-10-09 11:04:57 +02:00

deepspeed

Skip tests properly (#31308)

2024-06-26 21:59:08 +01:00

extended

Skip tests properly (#31308)

2024-06-26 21:59:08 +01:00

fixtures

Implementation of SuperPoint and AutoModelForKeypointDetection (#28966)

2024-03-19 14:43:02 +00:00

fsdp

Llama et al. / FSDP : Fix breaking change in 4.40 for FSDP (#31161)

2024-06-26 14:50:08 +01:00

generation

[whisper] static kv cache (#31166)

2024-07-02 13:24:15 +01:00

models

[whisper] static kv cache (#31166)

2024-07-02 13:24:15 +01:00

optimization

fix get_scheduler when name is warmup_stable_decay (#31128)

2024-05-30 15:25:43 +01:00

peft_integration

FIX [CI]: Fix failing tests for peft integration (#29330)

2024-02-29 03:56:16 +01:00

pipelines

Skip tests properly (#31308)

2024-06-26 21:59:08 +01:00

quantization

Skip tests properly (#31308)

2024-06-26 21:59:08 +01:00

repo_utils

Allow # Ignore copy (#27328)

2023-12-07 10:00:08 +01:00

sagemaker

Pass datasets trust_remote_code (#31406)

2024-06-17 17:29:13 +01:00

tokenization

Skip tests properly (#31308)

2024-06-26 21:59:08 +01:00

trainer

add gather_use_object arguments (#31514)

2024-06-28 13:50:27 +01:00

utils

Move some test files (tets/test_xxx_utils.py) to tests/utils (#31730)

2024-07-02 13:46:03 +02:00

__init__.py

GPU text generation: mMoved the encoded_prompt to correct device

2020-01-06 15:11:12 +01:00

test_backbone_common.py

Align backbone stage selection with out_indices & out_features (#27606)

2023-12-20 18:33:17 +00:00

test_configuration_common.py

Move some test files (tets/test_xxx_utils.py) to tests/utils (#31730)

2024-07-02 13:46:03 +02:00

test_feature_extraction_common.py

Split common test from core tests (#24284)

2023-06-15 07:30:24 -04:00

test_image_processing_common.py

Skip tests properly (#31308)

2024-06-26 21:59:08 +01:00

test_image_transforms.py

fix: center_crop occasionally outputs off-by-one dimension matrix (#30934)

2024-05-21 13:56:52 +01:00

test_modeling_common.py

Move some test files (tets/test_xxx_utils.py) to tests/utils (#31730)

2024-07-02 13:46:03 +02:00

test_modeling_flax_common.py

add sdpa to ViT [follow up of #29325] (#30555)

2024-05-16 10:56:11 +01:00

test_modeling_tf_common.py

Port IDEFICS to tensorflow (#26870)

2024-05-13 15:59:46 +01:00

test_pipeline_mixin.py

Skip tests properly (#31308)

2024-06-26 21:59:08 +01:00

test_processing_common.py

add initial design for uniform processors + align model (#31197)

2024-06-13 16:27:16 +02:00

test_sequence_feature_extraction_common.py

Fix typo (#25966)

2023-09-05 10:12:25 +02:00

test_tokenization_common.py

Skip tests properly (#31308)

2024-06-26 21:59:08 +01:00