Iterative generation using Input embeds and past_key_values (#35890)

* Iterative generation using input embeds

* ruff fix

* Added Testcase

* Updated comment

* ♻️ Refactored testcase

* Skip test for these models

* Continue generation using input embeds and cache

* Skip generate_continue_from_embeds test

* Refactor `prepare_input_for_generation` func

* Continue generation using input embeds and cache

* Modular changes fix

* Overwrite 'prepare_inputs_for_generation' function
This commit is contained in:
Yaswanth Gali
2025-02-06 15:36:05 +05:30
committed by GitHub
parent b5f327f350
commit 7aee036e54
18 changed files with 276 additions and 34 deletions

View File

@@ -325,6 +325,10 @@ class FuyuModelTest(ModelTesterMixin, GenerationTesterMixin, PipelineTesterMixin
def test_model_parallelism(self):
super().test_model_parallelism()
@unittest.skip(reason="Fuyu `prepare_inputs_for_generation` function doesn't have cache position.")
def test_generate_continue_from_inputs_embeds():
pass
@slow
@require_torch_accelerator