SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity

fix a hidden bug of GenerationConfig, now the generation_config.json can be loaded successfully (#28604)

Browse Source 
* fix a hidden bug of GenerationConfig

* keep `sort_keys=True` to maintain visibility

* Update src/transformers/generation/configuration_utils.py

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

* Update configuration_utils.py

in case `obj` is a list, check the items in the list

---------

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
This commit is contained in:
Zhenwei
2024-01-24 01:48:38 +08:00
committed by GitHub
parent ebc8f47bd9
commit 39c3c0a72a
1 changed files with 10 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

10
src/transformers/generation/configuration_utils.py
View File

@@ -909,6 +909,16 @@ class GenerationConfig(PushToHubMixin):
for metadata_field in METADATA_FIELDS: for metadata_field in METADATA_FIELDS:
config_dict.pop(metadata_field, None) config_dict.pop(metadata_field, None)
def convert_keys_to_string(obj):
if isinstance(obj, dict):
return {str(key): convert_keys_to_string(value) for key, value in obj.items()}
elif isinstance(obj, list):
return [convert_keys_to_string(item) for item in obj]
else:
return obj
config_dict = convert_keys_to_string(config_dict)
return json.dumps(config_dict, indent=2, sort_keys=True) + "\n" return json.dumps(config_dict, indent=2, sort_keys=True) + "\n"
def to_json_file(self, json_file_path: Union[str, os.PathLike], use_diff: bool = True): def to_json_file(self, json_file_path: Union[str, os.PathLike], use_diff: bool = True):