Xibin Bayes Zhou
45c7bfb157
Add evolla rebase main (#36232)
* add evolla
* adding protein encoder part
* add initial processing test
* save processor
* add docstring
* add evolla processor
* add two test
* change vision to protein
* change resampler to sequence_compressor
* change vision to protein
* initial update for llama
* add initial update for llamaForCausalLM
* add `test_processor`, `test_saprot_output`, `test_protein_encoder_output`
* change evolla, but still working on it
* add test_single_forward
* pass test_attention_outputs
* pass test_hidden_states_output
* pass test_save_load and test_from_pretrained_no_checkpoint
* pass test_cpu_offload
* skip some tests
* update new progress
* skip test_model_is_small
* pass test_model_weights_reload_no_missing_tied_weights
* pass test_model_get_set_embeddings
* pass test_cpu_offload
* skip test_resize_embeddings
* add pipeline_model_mapping
* remote old setUp
* pass processor save_pretrained and load_pretrained
* remove pooling layer
* pass test_inputs_embeds_matches_input_ids
* pass test_model_is_small
* pass test_attention_outputs
* pass test_initialization
* pass test_model_get_set_embeddings
* pass test_single_forward
* skip test_disk_offload_bin and test_disk_offload_safetensors
* fix most tests
* pass test_protein_encoder_output
* remove useless code
* add EvollaForProteinText2Text
* pass test_saprot_output
* pass all EvollaModelTest test and remove processor test
* add processor test to its own file
* skip is_training since esm skipped it and the saprot code causes error when setting is_training True
* pass processor tests
* solve all except config
* pass most cases
* change init
* add doc to `configuration_evolla.py`
* remove image_processing test
* remove extra processor test
* remove extra modules
* remove extra modules
* change all configs into one config
* pass all evolla test
* pass `make fixup`
* update short summary
* update Evolla-10B-hf
* pass check_dummies.py and check_code_quality
* fix `tests/models/auto/test_tokenization_auto.py::AutoTokenizerTest::test_model_name_edge_cases_in_mappings`
* remove dummy codes
* change format
* fix llava issue
* update format
* update to solve llama3 access issue
* update to make forward right
* solve processor save load problem from instructblip solution
* remove unexpected file
* skip `test_generation_tester_mixin_inheritance`
* add `test_single_forward_correct` and `test_inference_natural_language_protein_reasoning`
* add `modular_evolla.py`
* solved issue #36362
* run `make fixup`
* update modular
* solve float32 training
* add fix
* solve `utils/check_docstrings.py`
* update
* update
* update
* remove other files and replace sequential and einsum
* add use case in document
* update the models
* update model
* change some wrong code
* Update src/transformers/models/evolla/modular_evolla.py
Co-authored-by: Cyril Vallez <cyril.vallez@gmail.com>
* Update src/transformers/models/evolla/modular_evolla.py
Co-authored-by: Cyril Vallez <cyril.vallez@gmail.com>
* Update src/transformers/models/evolla/modular_evolla.py
Co-authored-by: Cyril Vallez <cyril.vallez@gmail.com>
* Update src/transformers/models/evolla/modular_evolla.py
Co-authored-by: Cyril Vallez <cyril.vallez@gmail.com>
* fix issues mentioned in PR
* update style and rearrange the placement
* fix return_dict argument issue
* solve SaProtConfig issue
* Solve EvollaSaProtRotaryEmbedding issue
* solve attention_mask issue
* solve almosst all issues
* make style
* update config
* remove unrelated pickle file
* delete pickle files
* fix config
* simplify a lot
* remove past k-v from encoder
* continue work
* style
* skip it from init
* fix init
* fix init
* simplify more
* fill in docstrings
* change test for generation
* skip test
* fix style
---------
Co-authored-by: Chenchen Han <13980209828@163.com>
Co-authored-by: Cyril Vallez <cyril.vallez@huggingface.co>
Co-authored-by: Cyril Vallez <cyril.vallez@gmail.com>
2025-07-25 19:11:57 +02:00
..
2025-07-08 11:53:21 +02:00
2025-06-16 14:21:18 -07:00
2025-05-28 09:19:09 -07:00
2025-06-13 11:07:09 +00:00
2025-06-24 15:05:29 +02:00
2025-06-05 14:36:54 -07:00
2025-06-13 11:07:09 +00:00
2025-07-22 10:53:16 +01:00
2025-06-13 11:07:09 +00:00
2025-06-16 10:46:30 -07:00
2025-06-18 16:01:25 -07:00
2025-03-03 10:33:46 -08:00
2025-05-27 11:51:41 -07:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-03-03 10:33:46 -08:00
2025-04-30 12:15:43 +01:00
2025-05-27 11:51:22 -07:00
2025-06-13 11:07:09 +00:00
2025-06-30 10:42:56 -07:00
2025-05-23 13:03:47 -07:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-20 13:46:19 -07:00
2025-03-03 10:33:46 -08:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-17 19:37:18 +01:00
2025-06-09 15:02:50 -07:00
2025-07-11 10:59:09 -07:00
2025-06-10 09:30:05 -07:00
2025-06-27 15:26:57 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-04-02 14:57:38 -07:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-04-30 12:15:43 +01:00
2025-06-13 11:07:09 +00:00
2025-06-26 14:40:45 -07:00
2025-06-26 14:40:45 -07:00
2025-06-02 12:58:01 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-03-03 10:33:46 -08:00
2025-05-27 17:03:55 +02:00
2025-06-13 11:07:09 +00:00
2025-06-17 10:30:03 -07:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-03-03 10:33:46 -08:00
2025-06-27 10:35:30 -07:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-07-09 17:04:28 +02:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-04-04 11:36:05 -07:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-17 19:37:18 +01:00
2025-07-03 10:02:58 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-05-01 08:54:22 -07:00
2025-04-30 12:15:43 +01:00
2025-06-12 10:26:50 -07:00
2025-07-08 11:44:29 +02:00
2025-04-14 16:24:01 +02:00
2025-06-25 11:38:25 +02:00
2025-06-13 11:07:09 +00:00
2025-06-18 17:33:29 +00:00
2025-06-13 11:07:09 +00:00
2025-07-22 10:53:16 +01:00
2025-06-13 11:07:09 +00:00
2025-04-30 12:15:43 +01:00
2025-06-13 11:07:09 +00:00
2025-07-23 19:39:27 +02:00
2025-07-11 11:23:08 -07:00
2025-07-02 12:25:26 +01:00
2025-07-25 17:36:18 +02:00
2025-07-25 17:36:18 +02:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-03-03 10:33:46 -08:00
2025-07-25 19:11:57 +02:00
2025-03-03 10:33:46 -08:00
2025-05-21 10:43:11 +02:00
2025-07-21 14:59:36 +02:00
2025-04-30 12:15:43 +01:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-03-03 10:33:46 -08:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-05-07 17:47:51 +02:00
2025-06-26 14:40:45 -07:00
2025-07-21 11:03:20 +02:00
2025-07-07 14:41:33 +02:00
2025-06-26 14:40:45 -07:00
2025-06-13 11:07:09 +00:00
2025-07-21 13:24:34 +02:00
2025-07-08 08:22:04 +02:00
2025-07-08 08:22:04 +02:00
2025-06-26 14:40:45 -07:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-07-11 10:59:41 -07:00
2025-06-13 11:07:09 +00:00
2025-06-04 09:56:47 -07:00
2025-03-03 10:33:46 -08:00
2025-03-03 10:33:46 -08:00
2025-03-03 10:33:46 -08:00
2025-03-03 10:33:46 -08:00
2025-03-03 10:33:46 -08:00
2025-06-13 11:07:09 +00:00
2025-06-26 14:40:45 -07:00
2025-06-13 11:07:09 +00:00
2025-05-06 06:47:43 +02:00
2025-06-13 11:07:09 +00:00
2025-02-03 20:06:03 +01:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-03-03 10:33:46 -08:00
2025-03-03 10:33:46 -08:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-23 14:17:25 +00:00
2025-06-23 14:17:25 +00:00
2025-03-03 10:33:46 -08:00
2025-07-16 12:40:22 -07:00
2025-03-03 10:33:46 -08:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-05-12 11:55:51 +02:00
2025-05-23 17:17:38 +02:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-25 16:09:00 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-07-07 15:56:57 -07:00
2025-06-13 11:07:09 +00:00
2025-07-10 16:07:33 +02:00
2025-07-15 12:40:50 -07:00
2025-06-13 11:07:09 +00:00
2025-06-26 14:40:45 -07:00
2025-06-26 14:40:45 -07:00
2025-06-26 14:40:45 -07:00
2025-06-26 14:40:45 -07:00
2025-06-13 11:07:09 +00:00
2025-07-09 11:32:40 -07:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-16 07:44:40 -07:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-03-03 10:33:46 -08:00
2025-07-11 11:23:08 -07:00
2025-07-21 14:59:36 +02:00
2025-07-09 10:23:03 -07:00
2025-06-13 11:07:09 +00:00
2025-07-23 02:47:47 +00:00
2025-07-23 02:47:47 +00:00
2025-06-13 11:07:09 +00:00
2025-05-15 10:44:19 +02:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-03-03 10:33:46 -08:00
2025-06-13 11:07:09 +00:00
2025-07-22 09:01:55 -07:00
2025-07-11 16:26:58 +00:00
2025-07-11 16:26:58 +00:00
2025-04-15 11:33:09 +01:00
2025-05-07 17:47:51 +02:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-04-30 12:15:43 +01:00
2025-05-28 09:20:19 -07:00
2025-07-07 13:34:59 -04:00
2025-06-27 14:40:24 +00:00
2025-06-13 11:07:09 +00:00
2025-07-21 16:40:22 -07:00
2025-04-30 12:15:43 +01:00
2025-06-12 10:27:17 -07:00
2025-03-03 10:33:46 -08:00
2025-06-13 11:07:09 +00:00
2025-03-03 10:33:46 -08:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-27 14:39:43 +00:00
2025-06-13 11:07:09 +00:00
2025-05-27 16:24:36 -07:00
2025-06-26 14:40:45 -07:00
2025-07-21 16:41:01 -07:00
2025-06-13 11:07:09 +00:00
2025-07-22 20:41:39 +00:00
2025-03-03 10:33:46 -08:00
2025-04-30 12:15:43 +01:00
2025-07-23 10:57:48 -07:00
2025-07-25 02:40:11 +00:00
2025-06-13 11:07:09 +00:00
2025-05-07 17:47:51 +02:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-26 13:54:48 -07:00
2025-06-12 10:27:07 -07:00
2025-06-13 11:07:09 +00:00
2025-07-11 11:07:32 +02:00
2025-03-03 10:33:46 -08:00
2025-06-26 14:40:45 -07:00
2025-07-14 10:35:17 -07:00
2025-06-26 14:40:45 -07:00
2025-06-13 11:07:09 +00:00
2025-03-03 10:33:46 -08:00
2025-06-13 11:07:09 +00:00
2025-07-11 16:26:58 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-04-28 11:56:32 +01:00
2025-05-07 17:47:51 +02:00
2025-06-13 11:07:09 +00:00
2025-06-26 14:40:45 -07:00
2025-06-26 14:40:45 -07:00
2025-06-26 14:40:45 -07:00
2025-03-31 09:50:49 +02:00
2025-03-31 09:50:49 +02:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-03-03 10:33:46 -08:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-03-03 10:33:46 -08:00
2025-06-13 11:07:09 +00:00
2025-06-13 12:02:44 -07:00
2025-06-17 11:02:18 -07:00
2025-05-23 16:27:56 -07:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-03-03 10:33:46 -08:00
2025-06-13 11:07:09 +00:00
2025-07-18 17:27:16 +00:00
2025-06-25 14:31:20 +00:00
2025-06-25 14:31:20 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-04-25 12:46:17 -07:00
2025-04-18 13:30:41 -07:00
2025-06-25 15:12:15 +00:00
2025-06-23 14:17:25 +00:00
2025-07-08 17:06:12 +02:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-03-03 10:33:46 -08:00
2025-06-26 14:40:45 -07:00
2025-07-15 12:40:26 -07:00
2025-06-26 10:13:06 -07:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-05-21 16:16:43 -07:00
2025-05-23 13:04:13 -07:00
2025-07-10 15:34:10 -07:00
2025-04-30 12:15:43 +01:00
2025-07-09 12:06:46 -07:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-03-03 10:33:46 -08:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-03-03 10:33:46 -08:00
2025-07-16 11:57:13 +00:00
2025-06-13 11:07:09 +00:00
2025-03-03 10:33:46 -08:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-18 09:38:58 -07:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-03-03 10:33:46 -08:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-17 19:37:18 +01:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-05-28 09:19:43 -07:00
2025-06-13 11:07:09 +00:00
2025-03-21 15:35:22 -07:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-07-02 07:56:29 -07:00
2025-04-15 13:16:05 -07:00
2025-06-13 11:07:09 +00:00
2025-06-13 17:56:15 +01:00
2025-07-25 12:09:44 +00:00
2025-06-13 11:07:09 +00:00
2025-03-03 10:33:46 -08:00
2025-06-13 11:07:09 +00:00
2025-07-08 17:06:12 +02:00
2025-06-13 11:07:09 +00:00
2025-05-22 09:16:38 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-09 13:00:38 -07:00
2025-06-09 12:26:31 -07:00
2025-06-13 11:07:09 +00:00
2025-06-09 12:26:23 -07:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-06-13 11:07:09 +00:00
2025-03-03 10:33:46 -08:00
2025-07-23 11:00:25 -07:00
2025-06-13 11:07:09 +00:00
2025-03-03 10:33:46 -08:00
2025-03-03 10:33:46 -08:00
2025-06-04 22:59:17 +00:00