Add tie_weights() to LM heads and set bias in set_output_embeddings() (#28948)

* Add tie_weights() to LM heads and set bias in set_output_embeddings()

The bias were not tied correctly in some LM heads, and this change should fix that.

* Moving test_save_and_load_low_cpu_mem_usage to ModelTesterMixin

* Adding _tie_weights() to MPNet and Vilt

* Skip test for low cpu mem usage for Deta/DeformableDetr since they cannot init on meta device

* Rename to test name to save_load to match the convention
This commit is contained in:
JB (Don)
2024-02-15 04:39:01 +08:00
committed by GitHub
parent 3f4e79d29c
commit 725f4ad1cc
20 changed files with 104 additions and 0 deletions

View File

@@ -564,6 +564,10 @@ class DeformableDetrModelTest(ModelTesterMixin, GenerationTesterMixin, PipelineT
msg=f"Parameter {name} of model {model_class} seems not properly initialized",
)
@unittest.skip("Cannot be initialized on meta device as some weights are modified during the initialization")
def test_save_load_low_cpu_mem_usage(self):
pass
def test_two_stage_training(self):
model_class = DeformableDetrForObjectDetection
config, inputs_dict = self.model_tester.prepare_config_and_inputs_for_common()