SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity

Iterative generation using Input embeds and past_key_values (#35890)

Browse Source 
* Iterative generation using input embeds

* ruff fix

* Added Testcase

* Updated comment

* ♻️ Refactored testcase

* Skip test for these models

* Continue generation using input embeds and cache

* Skip generate_continue_from_embeds test

* Refactor `prepare_input_for_generation` func

* Continue generation using input embeds and cache

* Modular changes fix

* Overwrite 'prepare_inputs_for_generation' function
This commit is contained in:
Yaswanth Gali
2025-02-06 15:36:05 +05:30
committed by GitHub
parent b5f327f350
commit 7aee036e54
18 changed files with 276 additions and 34 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
tests/models/gpt_bigcode/test_modeling_gpt_bigcode.py
View File

@@ -450,6 +450,10 @@ class GPTBigCodeModelTest(ModelTesterMixin, GenerationTesterMixin, PipelineTeste
def test_past_key_values_format(self):
pass
@unittest.skip(reason="BigCodeGPT has a non-standard KV cache format and breaks this test.")
def test_generate_continue_from_inputs_embeds(self):
pass
def test_gpt_bigcode_model(self):
config_and_inputs = self.model_tester.prepare_config_and_inputs()
self.model_tester.create_and_check_gpt_bigcode_model(*config_and_inputs)