SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity

Generate: use GenerationConfig as the basis for .generate() parametrization (#20388)

Browse Source 
* generate from config mvp

* fix failing tests

* max_time test

* Load default gen config at model load time; Update docs

* further documentation; add tests

* adapt rag to the new structure

* handle models not instantiated with from_pretained (like in tests)

* better default generation config

* add can_generate fn

* handle legacy use case of ad hoc model config changes

* initialize gen config from config in individual methods, if gen config is none

* fix _get_decoder_start_token_id when called outside GenerationMixin

* correct model config load order (set attr > model config > decoder config)

* update rag to match latest changes

* Apply suggestions from code review

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* load gen config from model config in model.from_pretrained

* fix can_generate fn

* handle generate calls without a previous from_pretrained (e.g. tests)

* add legacy behavior (and a warning)

* lower logger severity

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
This commit is contained in:
Joao Gante
2022-12-15 18:27:20 +00:00
committed by GitHub
parent b1706f6908
commit 4bc723f87d
6 changed files with 694 additions and 733 deletions
Download Patch File Download Diff File Expand all files Collapse all files

33
tests/generation/test_configuration_utils.py
View File

@@ -13,11 +13,12 @@
# See the License for the specific language governing permissions and
# limitations under the License.
import copy
import tempfile
import unittest
from parameterized import parameterized
from transformers.generation import GenerationConfig
from transformers import AutoConfig, GenerationConfig
class LogitsProcessorTest(unittest.TestCase):
@@ -43,3 +44,33 @@ class LogitsProcessorTest(unittest.TestCase):
self.assertEqual(loaded_config.top_k, 50)
self.assertEqual(loaded_config.max_length, 20)
self.assertEqual(loaded_config.max_time, None)
def test_from_model_config(self):
model_config = AutoConfig.from_pretrained("gpt2")
generation_config_from_model = GenerationConfig.from_model_config(model_config)
default_generation_config = GenerationConfig()
# The generation config has loaded a few non-default parameters from the model config
self.assertNotEqual(generation_config_from_model, default_generation_config)
# One of those parameters is eos_token_id -- check if it matches
self.assertNotEqual(generation_config_from_model.eos_token_id, default_generation_config.eos_token_id)
self.assertEqual(generation_config_from_model.eos_token_id, model_config.eos_token_id)
def test_update(self):
generation_config = GenerationConfig()
update_kwargs = {
"max_new_tokens": 1024,
"foo": "bar",
}
update_kwargs_copy = copy.deepcopy(update_kwargs)
unused_kwargs = generation_config.update(**update_kwargs)
# update_kwargs was not modified (no side effects)
self.assertEqual(update_kwargs, update_kwargs_copy)
# update_kwargs was used to update the config on valid attributes
self.assertEqual(generation_config.max_new_tokens, 1024)
# `.update()` returns a dictionary of unused kwargs
self.assertEqual(unused_kwargs, {"foo": "bar"})