Yuxuan Zhang af9870265e GLM-4.1V Model support (#38431) ...

* 20250508 Model Architecture

* Update modeling_glm4v.py

* Update modeling_glm4v.py

* Update modeling_glm4v.py

* update 1447

* 0526

* update

* format

* problem

* update

* update with only image embed diff

* Final

* upload

* update

* 1

* upload with ruff

* update

* update

* work

* 1

* 1

* update with new note

* 2

* Update convert_glm4v_mgt_weights_to_hf.py

* Update tokenization_auto.py

* update with new format

* remove rmsnrom

* draft with videos

* draft

* update

* update

* fix for review problem

* try to remove min_pixel

* update

* for test

* remove timestamps

* remove item

* update with remove

* change

* update 2200

* update

* Delete app.py

* format

* update

* Update test_video_processing_glm4v.py

* 1

* 2

* use new name

* Update test_video_processing_glm4v.py

* remove docs

* change

* update for image processors update

* 2108

* 2128

* Update modular_glm4v.py

* 1

* update some

* update

* rename

* 1

* remove tests output

* 2

* add configuration

* update

* Update test_video_processing_glm4v.py

* fix simple forward tests

* update with modular

* 1

* fix more tests

* fix generation test

* fix beam search and init

* modular changed

* fix beam search in case of single-image/video. Fails if multiple visuals per text

* update processor

* update test

* pass

* fix beam search

* update

* param correct

* Update convert_glm4v_mgt_weights_to_hf.py

* 1

* Update test_modeling_glm4v.py

* 4

* 2

* 2123 video process

* 2

* revert

* 1

* 2

* revert processing

* update preprocesor

* changed

* 1

* update

* update

* 6

* update

* update

* update

* Delete tmp.txt

* config

* Update video_processing_glm4v.py

* apply modular correctly

* move functions

* fix order

* update the longest_edge

* style

* simplify a lot

* fix random order of classes

* skip integration tests

* correctly fix the tests

* fix TP plan

---------

Co-authored-by: raushan <raushan@huggingface.co>
Co-authored-by: Cyril Vallez <cyril.vallez@huggingface.co>
Co-authored-by: Cyril Vallez <cyril.vallez@gmail.com>

2025-06-25 10:43:05 +02:00

..

internal

Remove all traces of low_cpu_mem_usage (#38792)

2025-06-12 16:39:33 +02:00

main_classes

Use HF papers (#38184)

2025-06-13 11:07:09 +00:00

model_doc

GLM-4.1V Model support (#38431)

2025-06-25 10:43:05 +02:00

quantization

Use HF papers (#38184)

2025-06-13 11:07:09 +00:00

reference

Enhance Model Loading By Providing Parallelism, Uses Optional Env Flag (#36835)

2025-05-23 16:39:47 +00:00

tasks

No more Tuple, List, Dict (#38797)

2025-06-17 19:37:18 +01:00

_config.py

…

_redirects.yml

…

_toctree.yml

GLM-4.1V Model support (#38431)

2025-06-25 10:43:05 +02:00

accelerate.md

change fsdp_strategy to fsdp in TrainingArguments in accelerate doc (#38807)

2025-06-13 15:32:40 +00:00

accelerator_selection.md

[docs] add xpu environment variable for gpu selection (#38194)

2025-05-30 16:05:07 +00:00

add_new_model.md

No more Tuple, List, Dict (#38797)

2025-06-17 19:37:18 +01:00

add_new_pipeline.md

[docs] Redesign (#31757)

2025-03-03 10:33:46 -08:00

agents.md

[agents] remove agents 🧹 (#37368)

2025-04-11 18:42:37 +01:00

attention_interface.md

No more Tuple, List, Dict (#38797)

2025-06-17 19:37:18 +01:00

auto_docstring.md

[AutoDocstring] Based on inspect parsing of the signature (#33771)

2025-05-08 17:46:07 -04:00

backbones.md

[docs] Redesign (#31757)

2025-03-03 10:33:46 -08:00

cache_explanation.md

[docs] Format fix (#38414)

2025-06-03 09:53:23 -07:00

chat_extras.md

Update chat_extras.md with content correction (#36599)

2025-03-07 13:09:02 +00:00

chat_templating_multimodal.md

[chat-template] Unify tests and clean up 🧼 (#37275)

2025-04-10 14:42:32 +02:00

chat_templating_writing.md

[docs] Redesign (#31757)

2025-03-03 10:33:46 -08:00

chat_templating.md

[docs] Redesign (#31757)

2025-03-03 10:33:46 -08:00

community.md

…

contributing.md

…

conversations.md

[chat] generate parameterization powered by GenerationConfig and UX-related changes (#38047)

2025-05-12 14:04:41 +01:00

custom_models.md

No more Tuple, List, Dict (#38797)

2025-06-17 19:37:18 +01:00

debugging.md

[docs] Redesign (#31757)

2025-03-03 10:33:46 -08:00

deepspeed.md

chore: Fix typos in docs and examples (#36524)

2025-03-04 13:47:41 +00:00

executorch.md

[docs] Redesign (#31757)

2025-03-03 10:33:46 -08:00

fast_tokenizers.md

[docs] Redesign (#31757)

2025-03-03 10:33:46 -08:00

feature_extractors.md

[docs] Redesign (#31757)

2025-03-03 10:33:46 -08:00

fsdp.md

[docs] Redesign (#31757)

2025-03-03 10:33:46 -08:00

generation_features.md

chore: Fix typos in docs and examples (#36524)

2025-03-04 13:47:41 +00:00

generation_strategies.md

Fix custom generate from local directory (#38916)

2025-06-20 17:36:57 +01:00

gguf.md

Fix gguf docs (#36601)

2025-03-11 15:29:14 +01:00

glossary.md

Use HF papers (#38184)

2025-06-13 11:07:09 +00:00

how_to_hack_models.md

[doc] fix bugs in how_to_hack_models.md (#38198)

2025-05-19 10:37:54 -07:00

hpo_train.md

[Nit] Add Note on SigOpt being in Public Archive Mode (#38610)

2025-06-05 14:07:23 -07:00

image_processors.md

🔴 Video processors as a separate class (#35206)

2025-05-12 11:55:51 +02:00

index.md

[docs] Update docs moved to the course (#38800)

2025-06-13 12:02:27 -07:00

installation.md

byebye torch 2.0 (#37277)

2025-04-07 15:19:47 +02:00

kv_cache.md

[docs] update cache docs with new info (#38775)

2025-06-13 07:10:56 +00:00

llm_optims.md

[CI] green llama tests (#37244)

2025-04-03 14:15:53 +01:00

llm_tutorial_optimization.md

Use HF papers (#38184)

2025-06-13 11:07:09 +00:00

llm_tutorial.md

No more Tuple, List, Dict (#38797)

2025-06-17 19:37:18 +01:00

model_memory_anatomy.md

Use HF papers (#38184)

2025-06-13 11:07:09 +00:00

model_sharing.md

[docs] Redesign (#31757)

2025-03-03 10:33:46 -08:00

models.md

Remove all traces of low_cpu_mem_usage (#38792)

2025-06-12 16:39:33 +02:00

modular_transformers.md

[modular] CLI allows positional arguments, and more defaults names for the optional arg (#38979)

2025-06-23 12:40:01 +02:00

notebooks.md

…

optimizers.md

[docs] Redesign (#31757)

2025-03-03 10:33:46 -08:00

pad_truncation.md

…

peft.md

[docs] Redesign (#31757)

2025-03-03 10:33:46 -08:00

perf_hardware.md

chore: Fix typos in docs and examples (#36524)

2025-03-04 13:47:41 +00:00

perf_infer_cpu.md

remove ipex_optimize_model usage (#38632)

2025-06-06 20:04:44 +02:00

perf_infer_gpu_multi.md

Fix: make docs work better with doc builder (#38213)

2025-05-20 08:23:03 +00:00

perf_infer_gpu_one.md

Small typo lines 47 and 199 perf_infer_gpu_one.md (#37938)

2025-05-06 14:32:55 +01:00

perf_torch_compile.md

[docs] Redesign (#31757)

2025-03-03 10:33:46 -08:00

perf_train_cpu_many.md

remove ipex_optimize_model usage (#38632)

2025-06-06 20:04:44 +02:00

perf_train_cpu.md

remove ipex_optimize_model usage (#38632)

2025-06-06 20:04:44 +02:00

perf_train_gaudi.md

Add Intel Gaudi doc (#37855)

2025-04-29 13:28:06 -07:00

perf_train_gpu_many.md

docs: fix typo (#37567)

2025-04-17 14:54:44 +01:00

perf_train_gpu_one.md

[docs] Typos - Single GPU efficient training features (#38964)

2025-06-23 12:33:10 -07:00

perf_train_special.md

[docs] Redesign (#31757)

2025-03-03 10:33:46 -08:00

perf_train_tpu_tf.md

[docs] Redesign (#31757)

2025-03-03 10:33:46 -08:00

perplexity.md

…

philosophy.md

…

pipeline_gradio.md

[docs] Redesign (#31757)

2025-03-03 10:33:46 -08:00

pipeline_tutorial.md

chore: Fix typos in docs and examples (#36524)

2025-03-04 13:47:41 +00:00

pipeline_webserver.md

fix and enhance pipeline_webserver.md (#36992)

2025-04-15 08:35:05 -07:00

pr_checks.md

…

processors.md

[docs] add Audio import (#38195)

2025-05-19 13:16:35 +00:00

quicktour.md

Add Hugging Face authentication procedure for IDEs (PyCharm, VS Code,… (#38954)

2025-06-24 11:48:15 -07:00

run_scripts.md

Remove research projects (#36645)

2025-03-11 13:47:38 +00:00

serialization.md

[docs] Redesign (#31757)

2025-03-03 10:33:46 -08:00

serving.md

fix docs serving typos. (#37936)

2025-05-06 14:32:44 +01:00

testing.md

chore: Fix typos in docs and examples (#36524)

2025-03-04 13:47:41 +00:00

tf_xla.md

[docs] Redesign (#31757)

2025-03-03 10:33:46 -08:00

tflite.md

[docs] Redesign (#31757)

2025-03-03 10:33:46 -08:00

tokenizer_summary.md

Use HF papers (#38184)

2025-06-13 11:07:09 +00:00

tools.md

[agents] remove agents 🧹 (#37368)

2025-04-11 18:42:37 +01:00

torchscript.md

Fix wording in torchscript.md (#38004)

2025-05-08 16:47:45 +01:00

trainer.md

feat: add flexible Liger Kernel configuration to TrainingArguments (#38911)

2025-06-19 15:54:08 +00:00

training.md

[docs] Redesign (#31757)

2025-03-03 10:33:46 -08:00

troubleshooting.md

…

video_processors.md

🔴 Video processors as a separate class (#35206)

2025-05-12 11:55:51 +02:00