SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity

Fix typos in tests (#36547)

Browse Source 
Signed-off-by: co63oc <co63oc@users.noreply.github.com>
This commit is contained in:
co63oc
2025-03-06 07:04:06 +08:00
committed by GitHub
parent 752ef3fd4e
commit 996f512d52
99 changed files with 282 additions and 282 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
tests/test_tokenization_common.py
View File

@@ -838,7 +838,7 @@ class TokenizerTesterMixin:
toks_after_adding = tokenizer.tokenize(text)
toks_after_adding2 = tokenizer.tokenize(text2)
# Rust tokenizers dont't lowercase added tokens at the time calling `tokenizer.add_tokens`,
# Rust tokenizers don't lowercase added tokens at the time calling `tokenizer.add_tokens`,
# while python tokenizers do, so new_toks 0 and 2 would be treated as the same, so do new_toks 1 and 3.
self.assertIn(added, [2, 4])