SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity

[tests] add the missing require_sacremoses decorator (#29504)

Browse Source 
* add sacremoses check

* fix style

* for FlaubertTokenizer

* HerbertTokenizer fix

* add typeHint

* Update src/transformers/testing_utils.py

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

* make less skipped

* make quality

* remove import

---------

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
This commit is contained in:
Fanli Lin
2024-03-08 18:13:54 +08:00
committed by GitHub
parent bc764f4263
commit 8e589c83b6
6 changed files with 18 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
tests/models/herbert/test_tokenization_herbert.py
View File

@@ -20,11 +20,12 @@ import unittest
from transformers import HerbertTokenizer, HerbertTokenizerFast
from transformers.models.herbert.tokenization_herbert import VOCAB_FILES_NAMES
from transformers.testing_utils import get_tests_dir, require_tokenizers, slow
from transformers.testing_utils import get_tests_dir, require_sacremoses, require_tokenizers, slow
from ...test_tokenization_common import TokenizerTesterMixin
@require_sacremoses
@require_tokenizers
class HerbertTokenizationTest(TokenizerTesterMixin, unittest.TestCase):
tokenizer_class = HerbertTokenizer