SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity

enable misc cases on XPU & use device agnostic APIs for cases in tests (#38192)

Browse Source 
* use device agnostic APIs in tests

Signed-off-by: Matrix Yao <matrix.yao@intel.com>

* more

Signed-off-by: Matrix Yao <matrix.yao@intel.com>

* fix style

Signed-off-by: Matrix Yao <matrix.yao@intel.com>

* add reset_peak_memory_stats API

Signed-off-by: YAO Matrix <matrix.yao@intel.com>

* update

---------

Signed-off-by: Matrix Yao <matrix.yao@intel.com>
Signed-off-by: YAO Matrix <matrix.yao@intel.com>
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
This commit is contained in:
Yao Matrix
2025-05-20 16:09:01 +08:00
committed by GitHub
parent dbc4b91db4
commit 3bd1c20149
13 changed files with 52 additions and 30 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
tests/models/cohere/test_tokenization_cohere.py
View File

@@ -17,7 +17,11 @@ import unittest
from functools import lru_cache
from transformers import CohereTokenizerFast
from transformers.testing_utils import require_jinja, require_tokenizers, require_torch_multi_gpu
from transformers.testing_utils import (
require_jinja,
require_tokenizers,
require_torch_multi_accelerator,
)
from ...test_tokenization_common import TokenizerTesterMixin, use_cache_if_possible
@@ -55,7 +59,7 @@ class CohereTokenizationTest(TokenizerTesterMixin, unittest.TestCase):
return CohereTokenizerFast.from_pretrained(pretrained_name, **kwargs)
# This gives CPU OOM on a single-gpu runner (~60G RAM). On multi-gpu runner, it has ~180G RAM which is enough.
@require_torch_multi_gpu
@require_torch_multi_accelerator
def test_torch_encode_plus_sent_to_model(self):
super().test_torch_encode_plus_sent_to_model()