* fix image_attention gate in idefics modeling
* update comment
* cleaner gating
* fix gate condition
* create attention gate once
* update comment
* update doc of cross-attention forward
* improve comment
* bring back no_images
* pass cross_attention_gate similarly to no_images gate
* add information on gate shape
* fix no_images placement
* make tests for gate
* take off no_images logic
* update test based on comments
* raise value error if cross_attention_gate is None
* send cross_attention_gate to device
* Revert "send cross_attention_gate to device"
This reverts commit 054f84228405bfa2e75fecc502f6a96dc83cdc0b.
* send cross_attention_gate to device
* fix device in test + nit
* fill hidden_states with zeros instead of multiplying with the gate
* style
* Update src/transformers/models/idefics/modeling_idefics.py
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
* Update src/transformers/models/idefics/modeling_idefics.py
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
---------
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
851a4f7088