Shuming Hu
bf607f6d3b
PerceptionLM ( #37878 )
...
* plm template
* A working plm with fixed image features
* hacked processor
* First version that reproduced PLM output using PE from timm.
* Simplify and fix tie_word_embeddings
* Use PIL resize. Simplify converstion.
* First version that works with video input.
* simplifed image preprocessing (not batched)
* Minor fixes after rebasing on main.
* Video processor based on new API.
* Revert to use _preprocess for image processor.
* refactor with modular
* fix tie_word_embedding
* Testing with timm PE
* check in missed converstion from modular to model.py
* First working version of PLM with Eva PE. PLM-1B and 3B outputs are exactly the same as before. PLM-8B output has some differences.
* address review comments
* Fixed batching if video and image examples mixed.
* Simplify PE configuration.
* Enable AutoModel for PerceptionEncoder.
* Update PE config style.
* update all headers
* Minor fixes.
* Move lm_head to PerceptionLMForConditionalGeneration.
Fix vit_G model specification.
* Fix for testing_modeling_perception_lm.py
* Image processing refactoring to use more common parts.
* Fix processor test.
* update tests to use model from hub
* More test fixes.
* integration test GT update after rebasing; probably due to video preprocessing
* update test media path to hub
* Stop tracking local scripts
* address some review comments
* refactor image processing.
* small fixes
* update documentation and minor fixes
* remove scripts
* Minor fix for CI
* Fix image processing
* CI and doc fix
* CI formatting fix
* ruff fix
* ruff formatting
* ran utils/sort_auto_mappings.py
* update docstring
* more docstring udpates
* add vision_input_type default fallback for image processing
* more verbose variable naming
* test update
* Remove PE and PEConfig use AutoModel(TimmWrapper) instead
* Minor cleanup.
* Minor Fix: remove any ref to PE. Ruff format and check.
* fix docstring
* Fix modular/model consistency.Improvex docstringfor .
* Fix PerceptionLMForConditionalGenerationModelTest
* ruff fix
* fix for check_repo
* minor formatting
* dummy size arg to fix for processor test.
* Update docstring for PerceptionLMConfig
* Minor fixes from review feedback.
* Revert some minor changes per reviewer feedback.
* update base_model_prefix
* address reviewer feedback
* fix comment in modeling file
* address reviewer feedback
* ruff format
* Pre-merge test update.
* reapply modular and fix checkpoint name
* processor test path
* use modular a bit more
* remove dead code
* add token decorator
---------
Co-authored-by: Cyril Vallez <cyril.vallez@huggingface.co >
Co-authored-by: Cyril Vallez <cyril.vallez@gmail.com >
2025-07-11 11:07:32 +02:00
..
2025-07-08 11:53:21 +02:00
2025-06-16 14:21:18 -07:00
2025-05-28 09:19:09 -07:00
2025-06-13 11:07:09 +00:00
2025-06-24 15:05:29 +02:00
2025-06-05 14:36:54 -07:00
2025-06-13 11:07:09 +00:00
2025-06-26 11:04:23 +00:00
2025-06-13 11:07:09 +00:00
2025-06-16 10:46:30 -07:00
2025-06-18 16:01:25 -07:00
2025-03-03 10:33:46 -08:00
2025-05-27 11:51:41 -07:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-03-03 10:33:46 -08:00
2025-04-30 12:15:43 +01:00
2025-05-27 11:51:22 -07:00
2025-06-13 11:07:09 +00:00
2025-06-30 10:42:56 -07:00
2025-05-23 13:03:47 -07:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-20 13:46:19 -07:00
2025-03-03 10:33:46 -08:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-17 19:37:18 +01:00
2025-06-09 15:02:50 -07:00
2025-06-13 11:07:09 +00:00
2025-06-10 09:30:05 -07:00
2025-06-27 15:26:57 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-04-02 14:57:38 -07:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-04-30 12:15:43 +01:00
2025-06-13 11:07:09 +00:00
2025-06-26 14:40:45 -07:00
2025-06-26 14:40:45 -07:00
2025-06-02 12:58:01 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-03-03 10:33:46 -08:00
2025-05-27 17:03:55 +02:00
2025-06-13 11:07:09 +00:00
2025-06-17 10:30:03 -07:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-03-03 10:33:46 -08:00
2025-06-27 10:35:30 -07:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-07-09 17:04:28 +02:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-04-04 11:36:05 -07:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-17 19:37:18 +01:00
2025-07-03 10:02:58 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-05-01 08:54:22 -07:00
2025-04-30 12:15:43 +01:00
2025-06-12 10:26:50 -07:00
2025-07-08 11:44:29 +02:00
2025-04-14 16:24:01 +02:00
2025-06-25 11:38:25 +02:00
2025-06-13 11:07:09 +00:00
2025-06-18 17:33:29 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-04-30 12:15:43 +01:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-07-02 12:25:26 +01:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-03-03 10:33:46 -08:00
2025-03-03 10:33:46 -08:00
2025-05-21 10:43:11 +02:00
2025-04-30 12:15:43 +01:00
2025-04-30 12:15:43 +01:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-03-03 10:33:46 -08:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-05-07 17:47:51 +02:00
2025-06-26 14:40:45 -07:00
2025-05-07 17:47:51 +02:00
2025-07-07 14:41:33 +02:00
2025-06-26 14:40:45 -07:00
2025-06-13 11:07:09 +00:00
2025-07-08 08:22:04 +02:00
2025-07-08 08:22:04 +02:00
2025-06-26 14:40:45 -07:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-05-02 09:55:16 +02:00
2025-06-13 11:07:09 +00:00
2025-06-04 09:56:47 -07:00
2025-03-03 10:33:46 -08:00
2025-03-03 10:33:46 -08:00
2025-03-03 10:33:46 -08:00
2025-03-03 10:33:46 -08:00
2025-03-03 10:33:46 -08:00
2025-06-13 11:07:09 +00:00
2025-06-26 14:40:45 -07:00
2025-06-13 11:07:09 +00:00
2025-05-06 06:47:43 +02:00
2025-06-13 11:07:09 +00:00
2025-02-03 20:06:03 +01:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-03-03 10:33:46 -08:00
2025-03-03 10:33:46 -08:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-23 14:17:25 +00:00
2025-06-23 14:17:25 +00:00
2025-03-03 10:33:46 -08:00
2025-06-13 11:07:09 +00:00
2025-03-03 10:33:46 -08:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-05-12 11:55:51 +02:00
2025-05-23 17:17:38 +02:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-25 16:09:00 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-07-07 15:56:57 -07:00
2025-06-13 11:07:09 +00:00
2025-07-10 16:07:33 +02:00
2025-06-17 18:10:23 +02:00
2025-06-13 11:07:09 +00:00
2025-06-26 14:40:45 -07:00
2025-06-26 14:40:45 -07:00
2025-06-26 14:40:45 -07:00
2025-06-26 14:40:45 -07:00
2025-06-13 11:07:09 +00:00
2025-07-09 11:32:40 -07:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-16 07:44:40 -07:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-03-03 10:33:46 -08:00
2025-06-13 12:03:49 +00:00
2025-06-13 12:03:49 +00:00
2025-07-09 10:23:03 -07:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-05-15 10:44:19 +02:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-03-03 10:33:46 -08:00
2025-06-13 11:07:09 +00:00
2025-05-07 17:47:51 +02:00
2025-06-26 14:40:45 -07:00
2025-06-26 14:40:45 -07:00
2025-04-15 11:33:09 +01:00
2025-05-07 17:47:51 +02:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-04-30 12:15:43 +01:00
2025-05-28 09:20:19 -07:00
2025-07-07 13:34:59 -04:00
2025-06-27 14:40:24 +00:00
2025-06-13 11:07:09 +00:00
2025-04-30 12:15:43 +01:00
2025-06-12 10:27:17 -07:00
2025-03-03 10:33:46 -08:00
2025-06-13 11:07:09 +00:00
2025-03-03 10:33:46 -08:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-27 14:39:43 +00:00
2025-06-13 11:07:09 +00:00
2025-05-27 16:24:36 -07:00
2025-06-26 14:40:45 -07:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-03-03 10:33:46 -08:00
2025-04-30 12:15:43 +01:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-05-07 17:47:51 +02:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-26 13:54:48 -07:00
2025-06-12 10:27:07 -07:00
2025-06-13 11:07:09 +00:00
2025-07-11 11:07:32 +02:00
2025-03-03 10:33:46 -08:00
2025-06-26 14:40:45 -07:00
2025-04-17 23:08:24 +02:00
2025-06-26 14:40:45 -07:00
2025-06-13 11:07:09 +00:00
2025-03-03 10:33:46 -08:00
2025-06-13 11:07:09 +00:00
2025-03-03 10:33:46 -08:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-04-28 11:56:32 +01:00
2025-05-07 17:47:51 +02:00
2025-06-13 11:07:09 +00:00
2025-06-26 14:40:45 -07:00
2025-06-26 14:40:45 -07:00
2025-06-26 14:40:45 -07:00
2025-03-31 09:50:49 +02:00
2025-03-31 09:50:49 +02:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-03-03 10:33:46 -08:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-03-03 10:33:46 -08:00
2025-06-13 11:07:09 +00:00
2025-06-13 12:02:44 -07:00
2025-06-17 11:02:18 -07:00
2025-05-23 16:27:56 -07:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-03-03 10:33:46 -08:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-25 14:31:20 +00:00
2025-06-25 14:31:20 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-04-25 12:46:17 -07:00
2025-04-18 13:30:41 -07:00
2025-06-25 15:12:15 +00:00
2025-06-23 14:17:25 +00:00
2025-07-08 17:06:12 +02:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-03-03 10:33:46 -08:00
2025-06-26 14:40:45 -07:00
2025-06-13 11:07:09 +00:00
2025-06-26 10:13:06 -07:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-05-21 16:16:43 -07:00
2025-05-23 13:04:13 -07:00
2025-07-10 15:34:10 -07:00
2025-04-30 12:15:43 +01:00
2025-07-09 12:06:46 -07:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-03-03 10:33:46 -08:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-03-03 10:33:46 -08:00
2025-04-16 15:00:53 +02:00
2025-06-13 11:07:09 +00:00
2025-03-03 10:33:46 -08:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-18 09:38:58 -07:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-03-03 10:33:46 -08:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-17 19:37:18 +01:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-05-28 09:19:43 -07:00
2025-06-13 11:07:09 +00:00
2025-03-21 15:35:22 -07:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-07-02 07:56:29 -07:00
2025-04-15 13:16:05 -07:00
2025-06-13 11:07:09 +00:00
2025-06-13 17:56:15 +01:00
2025-06-13 11:07:09 +00:00
2025-03-03 10:33:46 -08:00
2025-06-13 11:07:09 +00:00
2025-07-08 17:06:12 +02:00
2025-06-13 11:07:09 +00:00
2025-05-22 09:16:38 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-09 13:00:38 -07:00
2025-06-09 12:26:31 -07:00
2025-06-13 11:07:09 +00:00
2025-06-09 12:26:23 -07:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-03-03 10:33:46 -08:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-03-03 10:33:46 -08:00
2025-03-03 10:33:46 -08:00
2025-06-04 22:59:17 +00:00