[VLMs] use only xxx_token_id for multimodal tokens (#37573)

* use only `xxx_token_id` for multimodal tokens * update modeling files as well * fixup * why fixup doesn't fix modular docstring first? * janus, need to update configs in the hub still * last fixup
2025-04-18 17:03:39 +02:00
parent 4afd3f4820
commit 2ba6b92a6f
63 changed files with 279 additions and 141 deletions
--- a/utils/check_config_attributes.py
+++ b/utils/check_config_attributes.py
@@ -351,8 +351,8 @@ def check_attribute_being_used(config_class, attributes, default_value, source_s
        "pad_index",
        "unk_index",
        "mask_index",
-        "image_token_index",  # for VLMs
-        "video_token_index",
+        "image_token_id",  # for VLMs
+        "video_token_id",
        "image_seq_length",
        "video_seq_length",
        "image_size",