Conversion from slow to fast for BPE spm vocabs contained an error. (#10120)

* Conversion from slow to fast for BPE spm vocabs contained an error.

- There is only 1 test currently (tokenizers + slow) that used the modified path
and it's reformer, which does not contain any ids modification so the
bug was silent for now.
- The real issue is that vocab variable was overloaded by
SentencePieceExtractor, leading to Slow specific vocab oddities to be
completely ignored
- The bug was reported here https://github.com/huggingface/transformers/issues/9518
- Ran the complete tokenization test suite with slow without error
(`RUN_SLOW=1 pytest -sv tests/test_tokenization_*`)

* Remove rebase error.

* Adding the fixture.

src/transformers/convert_slow_tokenizer.py

@@ -322,10 +322,11 @@ class SpmConverter(Converter):
         if model_type == 1:
             tokenizer = Tokenizer(Unigram(vocab, unk_id))
         elif model_type == 2:
-            vocab, merges = SentencePieceExtractor(self.original_tokenizer.vocab_file).extract()
+            _, merges = SentencePieceExtractor(self.original_tokenizer.vocab_file).extract()
+            bpe_vocab = {word: i for i, (word, score) in enumerate(vocab)}
             tokenizer = Tokenizer(
                 BPE(
-                    vocab,
+                    bpe_vocab,
                     merges,
                     unk_token=proto.trainer_spec.unk_piece,
                     fuse_unk=True,
@@ -424,9 +425,10 @@ class CamembertConverter(SpmConverter):
             ("<pad>", 0.0),
             ("</s>NOTUSED", 0.0),
             ("<unk>", 0.0),
+            ("<unk>NOTUSED", -100),
         ]
         # We down-grade the original SentencePiece by -100 to avoid using it and use our added token instead
-        vocab += [(piece.piece, piece.score if i != 0 else piece.score - 100) for i, piece in enumerate(proto.pieces)]
+        vocab += [(piece.piece, piece.score) for piece in proto.pieces[1:]]
         vocab += [("<mask>", 0.0)]
         return vocab