🚨Early-error🚨 config will error out if output_attentions=True and the attn implementation is wrong (#38288)

* Protect ParallelInterface

* early error out on output attention setting for no wraning in modeling

* modular update

* fixup

* update model tests

* update

* oups

* set model's config

* more cases

* ??

* properly fix

* fixup

* update

* last onces

* update

* fix?

* fix wrong merge commit

* fix hub test

* nits

* wow I am tired

* updates

* fix pipeline!

---------

Co-authored-by: Lysandre <hi@lysand.re>

tests/pipelines/test_pipelines_text_to_audio.py

@@ -259,6 +259,7 @@ class TextToAudioPipelineTests(unittest.TestCase):
         model_test_kwargs = {}
         if model.can_generate():  # not all models in this pipeline can generate and, therefore, take `generate` kwargs
             model_test_kwargs["max_new_tokens"] = 5
+        model.config._attn_implementation = "eager"
         speech_generator = TextToAudioPipeline(
             model=model,
             tokenizer=tokenizer,