NielsRogge
8490d3159c
Add ViTPose (#30530)
* First draft
* Make fixup
* Make forward pass worké
* Improve code
* More improvements
* More improvements
* Make predictions match
* More improvements
* Improve image processor
* Fix model tests
* Add classic decoder
* Convert classic decoder
* Verify image processor
* Fix classic decoder logits
* Clean up
* Add post_process_pose_estimation
* Improve post_process_pose_estimation
* Use AutoBackbone
* Add support for MoE models
* Fix tests, improve num_experts%
* Improve variable names
* Make fixup
* More improvements
* Improve post_process_pose_estimation
* Compute centers and scales
* Improve postprocessing
* More improvements
* Fix ViTPoseBackbone tests
* Add docstrings, fix image processor tests
* Update index
* Use is_cv2_available
* Add model to toctree
* Add cv2 to doc tests
* Remove script
* Improve conversion script
* Add coco_to_pascal_voc
* Add box_to_center_and_scale to image_transforms
* Update tests
* Add integration test
* Fix merge
* Address comments
* Replace numpy by pytorch, improve docstrings
* Remove get_input_embeddings
* Address comments
* Move coco_to_pascal_voc
* Address comment
* Fix style
* Address comments
* Fix test
* Address comment
* Remove udp
* Remove comment
* [WIP] need to check if the numpy function is same as cv
* add scipy affine_transform
* Update src/transformers/models/vitpose/image_processing_vitpose.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* refactor convert
* add output_shape
* add atol 5e-2
* Use hf_hub_download in conversion script
* make box_to_center more applicable
* skipt test_get_set_embedding
* fix to accept array and fix CI
* add co-contributor
* make it to tensor type output
* add torch
* change to torch tensor
* add more test
* minor change
* CI test change
* import torch should be above ImageProcessor
* make style
* try not use torch in def
* Update src/transformers/models/vitpose/image_processing_vitpose.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update src/transformers/models/vitpose_backbone/configuration_vitpose_backbone.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update src/transformers/models/vitpose_backbone/modeling_vitpose_backbone.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update src/transformers/models/vitpose/modeling_vitpose.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* fix
* fix
* add caution
* make more detail about dataset_index
* Update src/transformers/models/vitpose/modeling_vitpose.py
Co-authored-by: Sangbum Daniel Choi <34004152+SangbumChoi@users.noreply.github.com>
* Update src/transformers/models/vitpose/image_processing_vitpose.py
Co-authored-by: Sangbum Daniel Choi <34004152+SangbumChoi@users.noreply.github.com>
* add docs
* Update docs/source/en/model_doc/vitpose.md
* Update src/transformers/models/vitpose/configuration_vitpose.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update src/transformers/__init__.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Revert "Update src/transformers/__init__.py"
This reverts commit 7ffa504450bb9dbccf9c7ea668441b98a1939d5c.
* change name
* Update src/transformers/models/vitpose/image_processing_vitpose.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update tests/models/vitpose/test_modeling_vitpose.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update docs/source/en/model_doc/vitpose.md
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/vitpose/modeling_vitpose.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/vitpose_backbone/modeling_vitpose_backbone.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/vitpose/image_processing_vitpose.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* move vitpose only function to image_processor
* raise valueerror when using timm backbone
* use out_indices
* Update src/transformers/models/vitpose/image_processing_vitpose.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* remove camel-case of def flip_back
* rename vitposeEstimatorOutput
* Update src/transformers/models/vitpose_backbone/modeling_vitpose_backbone.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* fix confused camelcase of MLP
* remove in-place logic
* clear scale description
* make consistent batch format
* docs update
* formatting docstring
* add batch tests
* test docs change
* Update src/transformers/models/vitpose/image_processing_vitpose.py
* Update src/transformers/models/vitpose/configuration_vitpose.py
* chagne ViT to Vit
* change to enable MoE
* make fix-copies
* Update docs/source/en/model_doc/vitpose.md
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* extract udp
* add more described docs
* simple fix
* change to accept target_size
* make style
* Update src/transformers/models/vitpose/image_processing_vitpose.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/vitpose/configuration_vitpose.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* change to `verify_backbone_config_arguments`
* Update docs/source/en/model_doc/vitpose.md
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* remove unnecessary copy
* make config immutable
* enable gradient checkpointing
* update inappropriate docstring
* linting docs
* split function for visibility
* make style
* check isinstances
* change to acceptable use_pretrained_backbone
* make style
* remove copy in docs
* Update src/transformers/models/vitpose_backbone/modeling_vitpose_backbone.py
Co-authored-by: Pavel Iakubovskii <qubvel@gmail.com>
* Update docs/source/en/model_doc/vitpose.md
Co-authored-by: Pavel Iakubovskii <qubvel@gmail.com>
* Update src/transformers/models/vitpose/modeling_vitpose.py
Co-authored-by: Pavel Iakubovskii <qubvel@gmail.com>
* simple fix + make style
* change input config of activation function to string
* Update docs/source/en/model_doc/vitpose.md
Co-authored-by: Pavel Iakubovskii <qubvel@gmail.com>
* tmp docs
* delete index.md
* make fix-copies
* simple fix
* change conversion to sam2/mllama style
* Update src/transformers/models/vitpose/image_processing_vitpose.py
Co-authored-by: Pavel Iakubovskii <qubvel@gmail.com>
* Update src/transformers/models/vitpose/image_processing_vitpose.py
Co-authored-by: Pavel Iakubovskii <qubvel@gmail.com>
* refactor convert
* add supervision
* Update src/transformers/models/vitpose_backbone/modeling_vitpose_backbone.py
Co-authored-by: Pavel Iakubovskii <qubvel@gmail.com>
* remove reduntant def
* seperate code block for visualization
* add validation for num_moe
* final commit
* add labels
* [run-slow] vitpose, vitpose_backbone
* Update src/transformers/models/vitpose/convert_vitpose_to_hf.py
Co-authored-by: Pavel Iakubovskii <qubvel@gmail.com>
* enable all conversion
* final commit
* [run-slow] vitpose, vitpose_backbone
* ruff check --fix
* [run-slow] vitpose, vitpose_backbone
* rename split module
* [run-slow] vitpose, vitpose_backbone
* fix pos_embed
* Simplify init
* Revert "fix pos_embed"
This reverts commit 2c56a4806e30bc9b5753b142fa04b913306c54ff.
* refactor single loop
* allow flag to enable custom model
* efficiency of MoE to not use unused experts
* make style
* Fix range -> arange to avoid warning
* Revert MOE router, a new one does not work
* Fix postprocessing a bit (labels)
* Fix type hint
* Fix docs snippets
* Fix links to checkpoints
* Fix checkpoints in tests
* Fix test
* Add image to docs
---------
Co-authored-by: Niels Rogge <nielsrogge@nielss-mbp.home>
Co-authored-by: Niels Rogge <nielsrogge@Nielss-MacBook-Pro.local>
Co-authored-by: sangbumchoi <danielsejong55@gmail.com>
Co-authored-by: Sangbum Daniel Choi <34004152+SangbumChoi@users.noreply.github.com>
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
Co-authored-by: Pavel Iakubovskii <qubvel@gmail.com>
2025-01-08 16:02:14 +00:00
..
2024-11-05 16:10:42 +01:00
2024-11-05 11:34:01 +01:00
2024-11-05 11:34:01 +01:00
2024-12-20 14:36:31 +01:00
2024-12-13 14:33:45 +01:00
2024-12-15 14:00:36 -05:00
2024-06-07 19:40:26 +01:00
2024-12-18 20:18:17 +01:00
2024-06-26 21:59:08 +01:00
2024-10-30 10:59:08 +00:00
2024-06-26 21:59:08 +01:00
2024-03-13 14:53:27 +01:00
2024-12-17 14:44:47 +01:00
2024-10-30 10:59:08 +00:00
2024-03-13 14:53:27 +01:00
2024-03-13 14:53:27 +01:00
2024-03-13 14:53:27 +01:00
2024-10-07 18:11:13 +02:00
2024-10-09 16:46:11 +01:00
2024-08-16 11:20:41 +01:00
2024-06-19 10:18:08 +01:00
2024-08-23 11:12:53 +01:00
2024-10-09 16:46:11 +01:00
2024-11-05 11:34:01 +01:00
2024-11-25 10:41:55 +01:00
2024-09-19 12:04:24 +02:00
2024-10-02 16:43:43 +02:00
2024-10-09 16:46:11 +01:00
2024-06-26 21:59:08 +01:00
2024-11-04 16:37:51 +01:00
2024-06-26 21:59:08 +01:00
2024-10-31 15:48:11 -04:00
2024-09-25 19:05:01 +01:00
2024-12-13 14:33:45 +01:00
2024-12-13 14:33:45 +01:00
2024-11-05 11:34:01 +01:00
2024-12-13 14:33:45 +01:00
2024-06-26 21:59:08 +01:00
2024-09-24 16:40:48 +01:00
2024-12-13 09:35:50 +01:00
2024-12-13 09:35:50 +01:00
2024-12-17 11:26:43 +01:00
2024-12-13 14:33:45 +01:00
2024-06-26 21:59:08 +01:00
2024-08-06 11:33:05 +01:00
2024-06-26 21:59:08 +01:00
2024-10-09 16:46:11 +01:00
2024-06-26 21:59:08 +01:00
2024-10-31 16:36:13 +01:00
2024-06-19 10:18:08 +01:00
2024-12-13 14:33:45 +01:00
2024-12-17 14:44:47 +01:00
2024-07-29 10:52:13 +01:00
2024-11-25 10:43:16 +01:00
2024-11-25 10:43:16 +01:00
2024-08-23 11:12:53 +01:00
2024-11-19 11:18:58 -05:00
2024-08-06 11:33:05 +01:00
2024-11-20 11:31:21 +01:00
2024-12-15 14:00:36 -05:00
2025-01-07 11:34:56 +01:00
2024-06-26 21:59:08 +01:00
2024-09-21 01:58:00 +01:00
2024-12-24 13:21:59 +01:00
2024-11-05 13:41:48 +01:00
2023-06-29 10:17:36 +01:00
2024-10-02 16:43:43 +02:00
2024-03-25 10:33:38 +01:00
2024-11-20 11:31:21 +01:00
2024-08-27 11:58:27 +01:00
2024-10-07 18:11:13 +02:00
2025-01-07 11:50:02 +01:00
2024-12-18 16:53:39 +01:00
2024-06-26 21:59:08 +01:00
2024-06-26 21:59:08 +01:00
2024-12-18 16:53:39 +01:00
2024-12-20 14:36:31 +01:00
2024-06-26 21:59:08 +01:00
2024-10-09 16:46:11 +01:00
2024-11-05 11:34:01 +01:00
2024-10-09 16:46:11 +01:00
2024-06-19 10:18:08 +01:00
2024-06-26 21:59:08 +01:00
2024-05-22 06:40:15 +02:00
2024-11-21 11:00:22 +01:00
2024-10-30 10:59:08 +00:00
2024-11-27 11:50:48 +01:00
2024-10-31 15:48:11 -04:00
2024-11-05 16:10:42 +01:00
2024-10-28 19:44:20 +01:00
2024-12-18 16:53:39 +01:00
2024-12-20 14:36:31 +01:00
2024-09-24 16:40:48 +01:00
2024-12-18 16:53:39 +01:00
2024-09-09 10:59:04 +02:00
2024-07-24 17:36:32 +01:00
2024-12-20 14:36:31 +01:00
2024-12-19 17:05:25 +01:00
2024-12-19 17:05:25 +01:00
2024-12-13 14:33:45 +01:00
2024-11-05 11:34:01 +01:00
2024-06-26 21:59:08 +01:00
2024-09-05 16:17:34 +02:00
2024-12-10 14:18:23 +01:00
2024-07-25 15:12:23 +02:00
2024-12-20 14:36:31 +01:00
2024-12-20 14:36:31 +01:00
2024-12-23 16:59:01 +01:00
2024-12-06 13:42:51 +00:00
2024-10-29 10:48:57 +01:00
2024-09-25 12:21:53 +01:00
2024-11-25 10:41:55 +01:00
2024-11-05 11:34:01 +01:00
2024-10-30 10:59:08 +00:00
2024-10-30 10:59:08 +00:00
2024-11-21 11:00:22 +01:00
2024-06-26 21:59:08 +01:00
2024-11-05 15:10:15 +00:00
2024-11-05 15:10:15 +00:00
2024-11-05 15:10:15 +00:00
2024-10-09 16:46:11 +01:00
2024-08-06 11:33:05 +01:00
2024-10-09 16:46:11 +01:00
2024-12-19 17:05:25 +01:00
2025-01-08 10:35:23 +01:00
2025-01-08 10:35:23 +01:00
2025-01-08 10:35:23 +01:00
2024-12-20 14:36:31 +01:00
2024-10-09 16:46:11 +01:00
2024-10-22 08:23:53 +02:00
2024-10-09 16:46:11 +01:00
2024-07-26 10:33:02 +02:00
2024-10-09 16:46:11 +01:00
2024-12-20 14:36:31 +01:00
2024-12-20 14:36:31 +01:00
2024-08-23 11:12:53 +01:00
2024-12-13 14:33:45 +01:00
2024-12-13 14:33:45 +01:00
2024-12-13 14:33:45 +01:00
2024-10-09 16:46:11 +01:00
2024-06-26 21:59:08 +01:00
2024-06-26 21:59:08 +01:00
2024-06-26 21:59:08 +01:00
2024-09-09 13:20:27 +01:00
2024-12-02 16:21:04 +01:00
2024-12-18 16:53:39 +01:00
2024-12-18 16:53:39 +01:00
2024-11-28 15:05:56 +01:00
2024-08-05 15:19:42 +01:00
2024-10-29 16:14:31 +01:00
2024-08-06 11:33:05 +01:00
2024-08-06 11:33:05 +01:00
2024-08-23 11:12:53 +01:00
2024-06-07 19:40:26 +01:00
2024-12-19 14:03:35 +01:00
2024-11-04 16:37:51 +01:00
2024-06-26 21:59:08 +01:00
2024-06-26 21:59:08 +01:00
2024-10-09 11:23:50 +01:00
2024-10-30 10:59:08 +00:00
2024-12-02 16:21:04 +01:00
2024-12-20 16:03:26 +01:00
2024-10-09 16:46:11 +01:00
2024-10-07 10:56:24 +02:00
2024-11-05 16:10:42 +01:00
2024-06-26 21:59:08 +01:00
2024-10-09 16:46:11 +01:00
2024-08-06 11:33:05 +01:00
2024-05-22 06:40:15 +02:00
2024-10-25 11:55:07 +01:00
2024-11-25 16:31:22 +01:00
2024-10-25 11:55:07 +01:00
2024-10-01 23:52:03 +02:00
2024-12-13 14:33:45 +01:00
2024-10-09 16:46:11 +01:00
2024-10-25 11:55:07 +01:00
2024-12-21 08:51:09 +00:00
2024-12-21 08:51:09 +00:00
2024-12-20 14:36:31 +01:00
2024-06-26 21:59:08 +01:00
2024-06-07 19:40:26 +01:00
2024-08-23 11:12:53 +01:00
2024-06-26 21:59:08 +01:00
2024-06-26 21:59:08 +01:00
2024-12-18 16:53:39 +01:00
2024-12-18 16:53:39 +01:00
2024-10-10 11:50:39 +02:00
2024-10-04 22:45:37 +02:00
2024-03-13 14:53:27 +01:00
2024-12-13 14:33:45 +01:00
2024-12-23 13:12:45 -05:00
2024-10-09 16:46:11 +01:00
2024-08-06 11:33:05 +01:00
2024-12-13 14:33:45 +01:00
2024-10-09 16:46:11 +01:00
2024-08-06 11:33:05 +01:00
2024-06-26 21:59:08 +01:00
2024-12-18 16:53:39 +01:00
2025-01-07 16:47:27 +01:00
2024-12-18 16:53:39 +01:00
2025-01-08 13:49:00 +01:00
2024-12-18 10:14:22 +01:00
2024-10-30 10:59:08 +00:00
2024-10-21 10:00:14 +02:00
2024-10-07 18:11:13 +02:00
2024-11-04 16:37:51 +01:00
2024-10-07 18:11:13 +02:00
2024-10-30 08:36:45 +00:00
2024-10-07 18:11:13 +02:00
2024-10-09 16:46:11 +01:00
2024-10-09 16:46:11 +01:00
2024-12-15 14:00:36 -05:00
2024-12-20 14:36:31 +01:00
2024-12-23 13:54:57 +01:00
2024-12-13 14:33:45 +01:00
2024-12-20 16:03:26 +01:00
2024-12-13 14:33:45 +01:00
2024-12-13 14:33:45 +01:00
2024-09-17 08:05:21 +02:00
2024-09-17 08:05:21 +02:00
2024-11-05 11:34:01 +01:00
2024-12-18 16:53:39 +01:00
2024-12-13 14:33:45 +01:00
2024-12-13 14:33:45 +01:00
2024-10-09 16:46:11 +01:00
2024-07-26 10:33:02 +02:00
2024-12-18 16:53:39 +01:00
2024-10-30 10:59:08 +00:00
2024-12-13 14:33:45 +01:00
2024-06-07 19:40:26 +01:00
2024-06-19 10:18:08 +01:00
2024-08-06 11:33:05 +01:00
2024-06-26 18:46:48 +01:00
2024-10-22 08:23:53 +02:00
2024-10-30 10:59:08 +00:00
2024-06-26 21:59:08 +01:00
2024-12-20 14:36:31 +01:00
2025-01-08 09:52:51 +01:00
2024-06-26 21:59:08 +01:00
2024-06-26 21:59:08 +01:00
2024-06-26 21:59:08 +01:00
2025-01-08 15:14:17 +00:00
2024-11-29 11:58:11 +00:00
2024-08-06 11:33:05 +01:00
2024-11-05 15:10:15 +00:00
2024-10-30 10:59:08 +00:00
2024-07-23 14:54:38 +08:00
2024-07-23 14:54:38 +08:00
2024-12-13 14:33:45 +01:00
2024-06-19 10:18:08 +01:00
2025-01-08 10:35:23 +01:00
2024-11-05 16:10:42 +01:00
2024-08-06 11:33:05 +01:00
2025-01-08 10:35:23 +01:00
2024-12-18 16:53:39 +01:00
2024-12-15 14:00:36 -05:00
2024-05-22 06:40:15 +02:00
2024-10-07 18:11:13 +02:00
2024-09-30 18:47:13 +02:00
2024-06-07 19:40:26 +01:00
2024-06-26 21:59:08 +01:00
2024-08-06 11:33:05 +01:00
2025-01-08 16:02:14 +00:00
2025-01-08 16:02:14 +00:00
2024-10-03 14:01:24 +01:00
2024-10-15 11:27:54 +02:00
2024-12-13 14:33:45 +01:00
2024-10-02 10:41:08 +02:00
2024-07-23 14:54:38 +08:00
2024-09-26 19:38:20 +02:00
2024-06-17 17:29:13 +01:00
2024-07-23 14:54:38 +08:00
2024-12-18 14:13:21 +01:00
2024-11-05 11:34:01 +01:00
2024-10-31 16:36:13 +01:00
2024-10-09 16:46:11 +01:00
2024-08-28 10:26:00 +02:00
2024-10-25 11:55:07 +01:00
2024-10-09 16:46:11 +01:00
2024-10-09 16:46:11 +01:00
2024-12-13 14:33:45 +01:00
2024-06-26 21:59:08 +01:00
2025-01-07 10:08:45 +01:00
2024-10-22 15:50:54 +02:00
2022-05-03 14:42:02 +02:00