SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity

Don't use default attn if pre-set in sub-config (#38526)

Browse Source 
* don't use default attn if pre-set in sib-config

* style

* add a test maybe
This commit is contained in:
Raushan Turganbay
2025-06-03 09:53:07 +02:00
committed by GitHub
parent bf68dd9e6e
commit 55ec319de6
2 changed files with 14 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

9
src/transformers/modeling_utils.py
View File

@@ -2277,8 +2277,13 @@ class PreTrainedModel(nn.Module, ModuleUtilsMixin, PushToHubMixin, PeftAdapterMi
if not isinstance(requested_attn_implementation, dict)
else requested_attn_implementation.get(key, None)
)
# For models with backbone sub-config might be not initialized
if sub_config is not None:
# For models with backbone sub-config might be not initialized. Set the requested att
# if the config hasn't got any attn pre-set and the requested attn in not `None` (i.e not the default attn)
if (
sub_config is not None
and sub_config._attn_implementation_internal is None
and curr_attn_implementation is not None
):
sub_config._attn_implementation_internal = curr_attn_implementation
if config._attn_implementation == "flash_attention_2":