Shuming Hu
bf607f6d3b
PerceptionLM ( #37878 )
...
* plm template
* A working plm with fixed image features
* hacked processor
* First version that reproduced PLM output using PE from timm.
* Simplify and fix tie_word_embeddings
* Use PIL resize. Simplify converstion.
* First version that works with video input.
* simplifed image preprocessing (not batched)
* Minor fixes after rebasing on main.
* Video processor based on new API.
* Revert to use _preprocess for image processor.
* refactor with modular
* fix tie_word_embedding
* Testing with timm PE
* check in missed converstion from modular to model.py
* First working version of PLM with Eva PE. PLM-1B and 3B outputs are exactly the same as before. PLM-8B output has some differences.
* address review comments
* Fixed batching if video and image examples mixed.
* Simplify PE configuration.
* Enable AutoModel for PerceptionEncoder.
* Update PE config style.
* update all headers
* Minor fixes.
* Move lm_head to PerceptionLMForConditionalGeneration.
Fix vit_G model specification.
* Fix for testing_modeling_perception_lm.py
* Image processing refactoring to use more common parts.
* Fix processor test.
* update tests to use model from hub
* More test fixes.
* integration test GT update after rebasing; probably due to video preprocessing
* update test media path to hub
* Stop tracking local scripts
* address some review comments
* refactor image processing.
* small fixes
* update documentation and minor fixes
* remove scripts
* Minor fix for CI
* Fix image processing
* CI and doc fix
* CI formatting fix
* ruff fix
* ruff formatting
* ran utils/sort_auto_mappings.py
* update docstring
* more docstring udpates
* add vision_input_type default fallback for image processing
* more verbose variable naming
* test update
* Remove PE and PEConfig use AutoModel(TimmWrapper) instead
* Minor cleanup.
* Minor Fix: remove any ref to PE. Ruff format and check.
* fix docstring
* Fix modular/model consistency.Improvex docstringfor .
* Fix PerceptionLMForConditionalGenerationModelTest
* ruff fix
* fix for check_repo
* minor formatting
* dummy size arg to fix for processor test.
* Update docstring for PerceptionLMConfig
* Minor fixes from review feedback.
* Revert some minor changes per reviewer feedback.
* update base_model_prefix
* address reviewer feedback
* fix comment in modeling file
* address reviewer feedback
* ruff format
* Pre-merge test update.
* reapply modular and fix checkpoint name
* processor test path
* use modular a bit more
* remove dead code
* add token decorator
---------
Co-authored-by: Cyril Vallez <cyril.vallez@huggingface.co >
Co-authored-by: Cyril Vallez <cyril.vallez@gmail.com >
2025-07-11 11:07:32 +02:00
..
2025-07-08 21:46:32 +02:00
2025-06-11 17:28:06 +01:00
2025-07-03 15:13:06 +02:00
2025-07-01 09:08:21 +00:00
2025-06-24 15:05:29 +02:00
2025-07-03 22:45:30 +02:00
2025-07-08 17:06:12 +02:00
2025-06-26 11:04:23 +00:00
2025-05-23 17:17:38 +02:00
2025-07-01 11:33:20 +00:00
2025-06-18 09:20:49 +02:00
2025-06-06 09:29:51 +02:00
2025-06-11 17:28:06 +01:00
2025-04-08 14:12:08 +02:00
2025-04-08 14:12:08 +02:00
2025-06-25 14:31:20 +00:00
2025-06-25 17:29:10 +00:00
2025-04-28 11:39:11 +01:00
2025-04-08 14:12:08 +02:00
2025-04-08 14:12:08 +02:00
2025-06-11 17:28:06 +01:00
2025-06-06 08:23:15 +00:00
2025-05-16 13:26:54 +02:00
2025-04-14 17:07:48 +02:00
2025-04-28 15:08:46 +02:00
2025-06-11 17:28:06 +01:00
2025-06-11 17:28:06 +01:00
2025-06-20 17:28:32 +02:00
2025-07-08 10:38:25 +02:00
2025-06-11 17:28:06 +01:00
2025-06-23 14:17:25 +00:00
2025-04-08 14:12:08 +02:00
2025-04-08 14:12:08 +02:00
2025-06-11 17:28:06 +01:00
2025-06-06 20:17:37 +02:00
2025-07-08 10:18:26 +02:00
2025-04-15 18:31:20 +02:00
2025-07-08 17:06:12 +02:00
2025-06-11 17:28:06 +01:00
2025-04-08 14:12:08 +02:00
2025-07-08 17:06:12 +02:00
2025-06-02 15:57:32 +02:00
2025-04-08 14:12:08 +02:00
2025-06-12 16:39:33 +02:00
2025-07-04 12:48:10 +02:00
2025-07-01 11:33:20 +00:00
2025-06-10 11:46:52 +02:00
2025-07-01 20:54:31 +02:00
2025-06-11 17:28:06 +01:00
2025-07-01 20:54:31 +02:00
2025-06-11 17:28:06 +01:00
2025-04-08 14:12:08 +02:00
2025-04-08 14:12:08 +02:00
2025-05-28 10:20:43 +02:00
2025-06-11 17:28:06 +01:00
2025-07-01 20:54:31 +02:00
2025-07-01 20:54:31 +02:00
2025-07-01 20:54:31 +02:00
2025-07-08 17:06:12 +02:00
2025-07-08 17:06:12 +02:00
2025-06-13 16:22:12 +01:00
2025-06-11 17:28:06 +01:00
2025-06-11 17:28:06 +01:00
2025-04-08 14:12:08 +02:00
2025-07-09 17:04:28 +02:00
2025-06-24 20:16:56 +02:00
2025-07-01 20:54:31 +02:00
2025-06-11 17:28:06 +01:00
2025-06-12 12:47:15 +02:00
2025-07-04 12:48:10 +02:00
2025-07-01 20:54:31 +02:00
2025-07-08 17:06:12 +02:00
2025-06-12 16:39:33 +02:00
2025-04-08 14:12:08 +02:00
2025-06-11 17:28:06 +01:00
2025-06-20 17:28:32 +02:00
2025-06-11 17:28:06 +01:00
2025-04-08 14:12:08 +02:00
2025-07-08 11:44:29 +02:00
2025-05-23 17:17:38 +02:00
2025-06-27 16:54:11 +02:00
2025-06-11 17:28:06 +01:00
2025-07-02 22:48:30 +02:00
2025-04-16 21:59:24 +02:00
2025-06-11 17:28:06 +01:00
2025-07-03 19:56:02 +02:00
2025-07-08 17:06:12 +02:00
2025-06-11 17:28:06 +01:00
2025-07-02 12:25:26 +01:00
2025-04-10 20:54:21 +02:00
2025-06-11 17:28:06 +01:00
2025-05-23 18:29:31 +01:00
2025-07-03 19:56:02 +02:00
2025-07-02 15:03:57 +02:00
2025-07-07 15:04:26 +02:00
2025-06-11 17:28:06 +01:00
2025-06-23 14:17:25 +00:00
2025-05-01 15:21:55 +02:00
2025-07-03 15:13:06 +02:00
2025-06-19 10:56:34 +02:00
2025-06-11 17:28:06 +01:00
2025-05-07 17:47:51 +02:00
2025-07-03 22:45:30 +02:00
2025-06-13 11:13:32 +02:00
2025-07-03 19:56:02 +02:00
2025-07-01 10:34:53 +02:00
2025-04-08 17:15:37 +01:00
2025-07-03 19:56:02 +02:00
2025-06-18 09:20:49 +02:00
2025-07-10 10:44:28 +02:00
2025-07-03 15:13:06 +02:00
2025-07-01 11:33:20 +00:00
2025-06-27 14:01:53 +02:00
2025-05-15 10:44:19 +02:00
2025-06-11 17:28:06 +01:00
2025-05-23 18:29:31 +01:00
2025-04-10 20:54:21 +02:00
2025-04-08 14:12:08 +02:00
2025-06-11 17:28:06 +01:00
2025-05-26 10:18:53 +02:00
2025-07-08 17:06:12 +02:00
2025-05-26 10:18:53 +02:00
2025-05-26 10:18:53 +02:00
2025-05-26 10:18:53 +02:00
2025-07-02 22:47:55 +02:00
2025-06-11 17:28:06 +01:00
2025-07-03 19:56:02 +02:00
2025-04-08 14:12:08 +02:00
2025-04-29 12:17:55 +01:00
2025-07-03 15:13:06 +02:00
2025-07-08 17:06:12 +02:00
2025-04-08 14:12:08 +02:00
2025-07-01 11:33:20 +00:00
2025-07-01 11:33:20 +00:00
2025-07-01 11:33:20 +00:00
2025-04-08 14:12:08 +02:00
2025-04-22 11:07:34 +01:00
2025-05-23 17:17:38 +02:00
2025-07-08 10:38:25 +02:00
2025-07-08 10:38:25 +02:00
2025-07-01 11:33:20 +00:00
2025-06-13 16:14:58 +02:00
2025-07-01 11:33:20 +00:00
2025-05-23 18:29:31 +01:00
2025-07-01 11:33:20 +00:00
2025-07-08 17:06:12 +02:00
2025-07-01 09:08:21 +00:00
2025-06-25 14:31:20 +00:00
2025-06-26 20:07:17 +02:00
2025-06-25 14:31:20 +00:00
2025-06-11 17:28:06 +01:00
2025-07-03 15:13:06 +02:00
2025-07-10 16:07:33 +02:00
2025-07-03 15:13:06 +02:00
2025-04-08 14:12:08 +02:00
2025-07-01 23:27:22 +02:00
2025-06-30 11:49:03 +02:00
2025-07-01 11:33:20 +00:00
2025-07-01 11:33:20 +00:00
2025-07-02 23:41:14 +02:00
2025-07-01 11:33:20 +00:00
2025-06-11 17:28:06 +01:00
2025-06-11 17:28:06 +01:00
2025-05-23 17:17:38 +02:00
2025-06-12 16:39:33 +02:00
2025-05-22 17:12:58 +02:00
2025-07-02 15:03:57 +02:00
2025-07-02 15:03:57 +02:00
2025-06-12 16:39:33 +02:00
2025-06-17 19:37:18 +01:00
2025-07-02 22:47:55 +02:00
2025-07-02 22:47:55 +02:00
2025-06-11 17:28:06 +01:00
2025-03-28 15:09:35 +01:00
2025-04-08 14:12:08 +02:00
2025-04-08 14:12:08 +02:00
2025-07-03 15:13:06 +02:00
2025-06-24 18:01:15 +02:00
2025-07-05 11:34:28 +02:00
2025-06-23 17:07:18 +02:00
2025-07-01 11:33:20 +00:00
2025-07-03 15:13:06 +02:00
2025-04-15 11:33:09 +01:00
2025-06-12 16:15:35 +02:00
2025-04-28 11:39:11 +01:00
2025-06-11 17:28:06 +01:00
2025-07-02 22:47:55 +02:00
2025-07-07 13:34:59 -04:00
2025-07-07 13:34:59 -04:00
2025-07-02 22:47:55 +02:00
2025-04-08 14:12:08 +02:00
2025-07-08 17:06:12 +02:00
2025-06-12 16:39:33 +02:00
2025-06-11 17:28:06 +01:00
2025-07-03 15:13:06 +02:00
2025-04-08 14:12:08 +02:00
2025-06-11 17:28:06 +01:00
2025-07-03 15:13:06 +02:00
2025-07-03 15:13:06 +02:00
2025-04-28 11:39:11 +01:00
2025-04-08 17:15:37 +01:00
2025-05-26 10:18:53 +02:00
2025-03-28 15:09:35 +01:00
2025-04-28 11:39:11 +01:00
2025-06-27 14:39:43 +00:00
2025-04-08 14:12:08 +02:00
2025-06-13 11:13:32 +02:00
2025-07-04 12:48:10 +02:00
2025-04-08 14:12:08 +02:00
2025-07-02 15:03:57 +02:00
2025-07-02 22:48:30 +02:00
2025-06-11 17:28:06 +01:00
2025-06-11 17:28:06 +01:00
2025-07-10 05:18:44 +00:00
2025-07-10 05:18:44 +00:00
2025-07-03 19:56:02 +02:00
2025-07-01 11:33:20 +00:00
2025-05-22 17:12:58 +02:00
2025-04-25 16:57:09 +02:00
2025-06-11 17:28:06 +01:00
2025-05-23 17:17:38 +02:00
2025-06-25 14:31:20 +00:00
2025-07-11 11:07:32 +02:00
2025-05-23 18:29:31 +01:00
2025-05-23 18:29:31 +01:00
2025-07-10 11:51:55 +02:00
2025-07-08 17:06:12 +02:00
2025-05-28 16:44:20 +01:00
2025-04-08 14:12:08 +02:00
2025-06-20 17:28:32 +02:00
2025-06-23 14:17:25 +00:00
2025-05-22 17:12:58 +02:00
2025-07-02 22:48:30 +02:00
2025-06-25 17:29:10 +00:00
2025-06-12 12:47:15 +02:00
2025-06-06 08:23:15 +00:00
2025-07-02 22:48:30 +02:00
2025-07-02 22:48:30 +02:00
2025-07-03 19:56:02 +02:00
2025-07-01 11:33:20 +00:00
2025-07-01 11:33:20 +00:00
2025-06-23 13:44:50 +02:00
2025-05-28 16:44:20 +01:00
2025-07-03 19:56:02 +02:00
2025-06-27 14:01:53 +02:00
2025-06-18 14:36:03 +02:00
2025-06-23 17:42:46 +02:00
2025-07-07 15:13:25 +02:00
2025-04-08 14:12:08 +02:00
2025-07-04 13:35:53 +02:00
2025-06-11 17:28:06 +01:00
2025-07-02 22:48:30 +02:00
2025-06-11 17:28:06 +01:00
2025-06-11 17:28:06 +01:00
2025-04-10 20:54:21 +02:00
2025-06-11 17:28:06 +01:00
2025-07-02 22:47:55 +02:00
2025-07-02 22:47:55 +02:00
2025-05-23 17:17:38 +02:00
2025-07-05 11:34:28 +02:00
2025-07-05 11:34:28 +02:00
2025-06-02 09:17:28 +00:00
2025-06-02 09:17:28 +00:00
2025-07-02 22:47:55 +02:00
2025-07-02 22:48:30 +02:00
2025-07-08 17:06:12 +02:00
2025-07-08 17:06:12 +02:00
2025-06-12 16:39:33 +02:00
2025-06-03 07:40:44 +00:00
2025-06-23 14:17:25 +00:00
2025-06-25 15:12:15 +00:00
2025-07-02 12:05:10 +02:00
2025-06-11 17:28:06 +01:00
2025-07-08 17:06:12 +02:00
2025-07-08 17:06:12 +02:00
2025-07-01 09:08:21 +00:00
2025-04-08 14:12:08 +02:00
2025-05-23 18:29:31 +01:00
2025-07-03 19:56:02 +02:00
2025-04-08 14:12:08 +02:00
2025-04-08 14:12:08 +02:00
2025-06-11 17:28:06 +01:00
2025-06-11 17:28:06 +01:00
2025-07-02 22:48:30 +02:00
2025-05-23 17:17:38 +02:00
2025-07-02 22:48:30 +02:00
2025-06-12 16:39:33 +02:00
2025-07-05 11:34:28 +02:00
2025-05-23 17:17:38 +02:00
2025-06-26 16:25:00 +01:00
2025-04-08 14:12:08 +02:00
2025-05-23 17:17:38 +02:00
2025-06-23 13:44:50 +02:00
2025-07-03 15:13:06 +02:00
2025-06-12 16:39:33 +02:00
2025-07-03 15:13:06 +02:00
2025-04-08 17:15:37 +01:00
2025-04-28 14:20:45 +01:00
2025-06-25 14:31:20 +00:00
2025-06-06 08:23:15 +00:00
2025-07-08 17:06:12 +02:00
2025-07-08 17:06:12 +02:00
2025-07-08 17:06:12 +02:00
2025-06-25 14:31:20 +00:00
2025-07-01 11:33:20 +00:00
2025-07-03 15:13:06 +02:00
2025-06-25 14:31:20 +00:00
2025-07-01 11:33:20 +00:00
2025-06-25 14:31:20 +00:00
2025-06-26 16:25:00 +01:00
2025-05-23 17:17:38 +02:00
2025-06-11 17:28:06 +01:00
2025-06-11 17:28:06 +01:00
2025-04-08 14:12:08 +02:00
2025-04-08 14:12:08 +02:00
2025-06-23 14:17:25 +00:00
2025-07-03 15:13:06 +02:00
2025-07-03 15:13:06 +02:00
2025-06-13 16:14:58 +02:00
2025-07-03 15:13:06 +02:00
2025-07-05 11:34:28 +02:00
2025-07-08 17:06:12 +02:00
2025-07-03 15:13:06 +02:00
2025-07-03 15:13:06 +02:00
2025-04-08 14:12:08 +02:00
2025-06-25 14:31:20 +00:00
2025-06-25 14:31:20 +00:00
2025-07-08 17:06:12 +02:00
2025-07-03 15:13:06 +02:00
2025-06-13 16:14:58 +02:00
2025-06-11 17:28:06 +01:00
2025-06-11 17:28:06 +01:00
2025-04-10 20:54:21 +02:00
2025-06-11 17:28:06 +01:00
2025-04-10 20:54:21 +02:00
2025-05-23 17:17:38 +02:00
2025-04-08 14:12:08 +02:00
2025-06-12 16:39:33 +02:00
2025-06-30 11:49:03 +02:00
2025-06-12 12:47:15 +02:00
2022-05-03 14:42:02 +02:00