Pavel Iakubovskii a957b7911a Add SigLIP 2 (#36323) ...

* Docs

* Inits

* Auto classes

* Add siglip base

* Add base tests

* Fix Siglip V1 for fix res version

* Add image processor

* Update conversion

* Experimenting with vectorized embeddings

* Fixup

* Add modular Siglip2Processor

* Add modular configuration

* Rename num patches

* Correct image and text features merging

* Working conversion script

* Refactoring conversion script

* Remove unused code in conversion script

* Shorten dict a bit

* Refactoring conversion

* Done conversion refactoring

* Fixup

* Modular siglip2

* Make model exportable and compilable without graph breaks

* Remove position_ids from image_processor

* REmove position ids from modeling file

* Update modular

* Type hint

* Fixup

* Set defaults to processor

* Add integration test

* Revert spatial shapes back to tensor

* Change order

* Fix most of the tests

* Fix docstring

* Remove interpolate_pos_encoding arg (not needed)

* Update docs

* Standardize processing

* Fix attention_mask in vision head

* Siglip v1: remove double transpose in FA2

* Update modular file

* Update FA2 test

* Update expected logits

* Fix interpolation for siglip2 image processor

* Skip init test

* Skip dispatch on flash test

* Fix modeling tests

* Fixup

* Add dummy objects

* Fix some docstrings

* Add siglip2 in index.md

* Fix consistency

* Add docs

* Remove size and data format

* Add image processor tests

* Fix

* Add fast image processor

* Fix style

* Fix

* Docs

* Set lowercase for tokenizer

* Adjust head size for Siglip v1

* Update siglip2 for consistency with siglip1

* Update siglip2 conversion

* Update pipeline

* Update checkpoints in tests

* Update checkpoint name

* Fix pooling for image classification model

* Fix FA2 test

* Update processor

* Fix check repo

* Update docs

* Fix typos

* Fix docstring for fast image processor

* Add siglip2 to FA2 docs

* Fix fast ip tests

* Fix constitency

* Fix tokenizer class for siglip v1

* Fix missing header

* Refactor scaling for clip, siglip, siglip2

* Remove unused imports

* Make fast IP default for siglip2

* Update docs

* Update checkpoints

* Update modular

* Update paper link

* Fixup

* Fix name in toctree

* Fix test

2025-02-21 09:04:19 +00:00

..

agents

use torch.testing.assertclose instead to get more details about error in cis (#35659)

2025-01-24 16:55:28 +01:00

bettertransformer

use torch.testing.assertclose instead to get more details about error in cis (#35659)

2025-01-24 16:55:28 +01:00

deepspeed

DeepSpeed github repo move sync (#36021)

2025-02-05 08:19:31 -08:00

extended

[tests] skip tests for xpu (#33553)

2024-09-19 19:28:04 +01:00

fixtures

Implementation of SuperPoint and AutoModelForKeypointDetection (#28966)

2024-03-19 14:43:02 +00:00

fsdp

[tests] make cuda-only tests device-agnostic (#35607)

2025-01-13 14:48:39 +01:00

generation

Remove flakiness in VLMs (#36242)

2025-02-18 11:41:07 +01:00

models

Add SigLIP 2 (#36323)

2025-02-21 09:04:19 +00:00

optimization

Support constant lr with cooldown (#35453)

2025-02-10 13:21:55 +01:00

peft_integration

use torch.testing.assertclose instead to get more details about error in cis (#35659)

2025-01-24 16:55:28 +01:00

pipelines

Add support for post-processing kwargs in image-text-to-text pipeline (#35374)

2025-02-18 17:43:36 -05:00

quantization

Fix TorchAoConfig not JSON serializable (#36206)

2025-02-18 11:05:42 +01:00

repo_utils

[Modular] skip modular checks based on diff (#36130)

2025-02-13 12:53:21 +00:00

sagemaker

Trainer - deprecate tokenizer for processing_class (#32385)

2024-10-02 14:08:46 +01:00

tokenization

Fix PretrainedTokenizerFast check => Fix PretrainedTokenizerFast Save (#35835)

2025-02-13 12:00:33 +01:00

tp

TP initialization module-by-module (#35996)

2025-02-19 14:04:57 +01:00

trainer

fix: prevent second save in the end of training if last step was saved already (#36219)

2025-02-20 17:38:52 +01:00

utils

Make cache traceable (#35873)

2025-02-20 09:59:25 +01:00

__init__.py

…

test_backbone_common.py

Align backbone stage selection with out_indices & out_features (#27606)

2023-12-20 18:33:17 +00:00

test_configuration_common.py

Load sub-configs from composite configs (#34410)

2024-11-05 11:34:01 +01:00

test_feature_extraction_common.py

Split common test from core tests (#24284)

2023-06-15 07:30:24 -04:00

test_image_processing_common.py

Au revoir flaky test_fast_is_faster_than_slow (#36240)

2025-02-17 18:30:07 +01:00

test_image_transforms.py

fix: center_crop occasionally outputs off-by-one dimension matrix (#30934)

2024-05-21 13:56:52 +01:00

test_modeling_common.py

Add SigLIP 2 (#36323)

2025-02-21 09:04:19 +00:00

test_modeling_flax_common.py

[tests] remove flax-pt equivalence and cross tests (#36283)

2025-02-19 15:13:27 +00:00

test_modeling_tf_common.py

[tests] remove flax-pt equivalence and cross tests (#36283)

2025-02-19 15:13:27 +00:00

test_pipeline_mixin.py

Add image text to text pipeline (#34170)

2024-10-31 15:48:11 -04:00

test_processing_common.py

Uniformize LlavaNextVideoProcessor kwargs (#35613)

2025-02-18 14:13:51 -05:00

test_sequence_feature_extraction_common.py

Fix typo (#25966)

2023-09-05 10:12:25 +02:00

test_tokenization_common.py

[tests] remove pt_tf equivalence tests (#36253)

2025-02-19 11:55:11 +00:00

test_training_args.py

CI: fix test-save-trainer (#36191)

2025-02-14 10:20:56 +01:00