Xibin Bayes Zhou
45c7bfb157
Add evolla rebase main (#36232)
* add evolla
* adding protein encoder part
* add initial processing test
* save processor
* add docstring
* add evolla processor
* add two test
* change vision to protein
* change resampler to sequence_compressor
* change vision to protein
* initial update for llama
* add initial update for llamaForCausalLM
* add `test_processor`, `test_saprot_output`, `test_protein_encoder_output`
* change evolla, but still working on it
* add test_single_forward
* pass test_attention_outputs
* pass test_hidden_states_output
* pass test_save_load and test_from_pretrained_no_checkpoint
* pass test_cpu_offload
* skip some tests
* update new progress
* skip test_model_is_small
* pass test_model_weights_reload_no_missing_tied_weights
* pass test_model_get_set_embeddings
* pass test_cpu_offload
* skip test_resize_embeddings
* add pipeline_model_mapping
* remote old setUp
* pass processor save_pretrained and load_pretrained
* remove pooling layer
* pass test_inputs_embeds_matches_input_ids
* pass test_model_is_small
* pass test_attention_outputs
* pass test_initialization
* pass test_model_get_set_embeddings
* pass test_single_forward
* skip test_disk_offload_bin and test_disk_offload_safetensors
* fix most tests
* pass test_protein_encoder_output
* remove useless code
* add EvollaForProteinText2Text
* pass test_saprot_output
* pass all EvollaModelTest test and remove processor test
* add processor test to its own file
* skip is_training since esm skipped it and the saprot code causes error when setting is_training True
* pass processor tests
* solve all except config
* pass most cases
* change init
* add doc to `configuration_evolla.py`
* remove image_processing test
* remove extra processor test
* remove extra modules
* remove extra modules
* change all configs into one config
* pass all evolla test
* pass `make fixup`
* update short summary
* update Evolla-10B-hf
* pass check_dummies.py and check_code_quality
* fix `tests/models/auto/test_tokenization_auto.py::AutoTokenizerTest::test_model_name_edge_cases_in_mappings`
* remove dummy codes
* change format
* fix llava issue
* update format
* update to solve llama3 access issue
* update to make forward right
* solve processor save load problem from instructblip solution
* remove unexpected file
* skip `test_generation_tester_mixin_inheritance`
* add `test_single_forward_correct` and `test_inference_natural_language_protein_reasoning`
* add `modular_evolla.py`
* solved issue #36362
* run `make fixup`
* update modular
* solve float32 training
* add fix
* solve `utils/check_docstrings.py`
* update
* update
* update
* remove other files and replace sequential and einsum
* add use case in document
* update the models
* update model
* change some wrong code
* Update src/transformers/models/evolla/modular_evolla.py
Co-authored-by: Cyril Vallez <cyril.vallez@gmail.com>
* Update src/transformers/models/evolla/modular_evolla.py
Co-authored-by: Cyril Vallez <cyril.vallez@gmail.com>
* Update src/transformers/models/evolla/modular_evolla.py
Co-authored-by: Cyril Vallez <cyril.vallez@gmail.com>
* Update src/transformers/models/evolla/modular_evolla.py
Co-authored-by: Cyril Vallez <cyril.vallez@gmail.com>
* fix issues mentioned in PR
* update style and rearrange the placement
* fix return_dict argument issue
* solve SaProtConfig issue
* Solve EvollaSaProtRotaryEmbedding issue
* solve attention_mask issue
* solve almosst all issues
* make style
* update config
* remove unrelated pickle file
* delete pickle files
* fix config
* simplify a lot
* remove past k-v from encoder
* continue work
* style
* skip it from init
* fix init
* fix init
* simplify more
* fill in docstrings
* change test for generation
* skip test
* fix style
---------
Co-authored-by: Chenchen Han <13980209828@163.com>
Co-authored-by: Cyril Vallez <cyril.vallez@huggingface.co>
Co-authored-by: Cyril Vallez <cyril.vallez@gmail.com>
2025-07-25 19:11:57 +02:00
..
2025-07-25 19:11:57 +02:00
2025-07-25 19:11:57 +02:00
2025-07-25 19:11:57 +02:00