Even more TF test fixes (#28146)

* Fix vision text dual encoder

* Small cleanup for wav2vec2 (not fixed yet)

* Small fix for vision_encoder_decoder

* Fix SAM builds

* Update TFBertTokenizer test with modern exporting + tokenizer

* Fix DeBERTa

* Fix DeBERTav2

* Try RAG fix but it's impossible to test locally

* Actually fix RAG now that I got FAISS working somehow

* Fix Wav2Vec2, add sermon

* Fix Hubert

tests/models/rag/test_modeling_tf_rag.py

@@ -1005,6 +1005,7 @@ class TFRagModelSaveLoadTests(unittest.TestCase):
                 retriever=rag_retriever,
                 config=rag_config,
             )
+            rag_sequence.build_in_name_scope()
             # check that the from pretrained methods work
             rag_sequence.save_pretrained(tmp_dirname)
             rag_sequence.from_pretrained(tmp_dirname, retriever=rag_retriever)
@@ -1056,6 +1057,7 @@ class TFRagModelSaveLoadTests(unittest.TestCase):
                 retriever=rag_retriever,
                 config=rag_config,
             )
+            rag_token.build_in_name_scope()
             # check that the from pretrained methods work
             rag_token.save_pretrained(tmp_dirname)
             rag_token.from_pretrained(tmp_dirname, retriever=rag_retriever)