SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity

Fix removing "cpu" from frozenset in bitsandbytes.py to allow better ROCm support. (#36975)

Browse Source 
* Fix removing "cpu" from frozenset in bitsandbytes.py to allow better ROCm support.

Related to https://github.com/bitsandbytes-foundation/bitsandbytes/issues/1573 and https://github.com/huggingface/transformers/issues/36949 , this resolves a bug in allowing ROCm/HIP support in bitsandbytes.

* Related to bitsandbytes-foundation/bitsandbytes#1573 and huggingface#36949 , this resolves a bug in the biteandbytes integration, allowing ROCm/HIP support in bitsandbytes.

---------

Co-authored-by: Mohamed Mekkouri <93391238+MekkCyber@users.noreply.github.com>
This commit is contained in:
Josh Marshall
2025-03-26 11:18:08 -04:00
committed by GitHub
parent 788e1092e9
commit 021006e1b0
1 changed files with 3 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
src/transformers/integrations/bitsandbytes.py
View File

@@ -496,7 +496,9 @@ def _validate_bnb_multi_backend_availability(raise_exception):
"You have Intel IPEX installed but if you're intending to use it for CPU, it might not have the right version. Be sure to double check that your PyTorch and IPEX installs are compatible."
)
available_devices.discard("cpu") # Only Intel CPU is supported by BNB at the moment
available_devices = frozenset(
[device for device in available_devices if device != "cpu"]
) # Only Intel CPU is supported by BNB at the moment
if not available_devices.intersection(bnb_supported_devices):
if raise_exception: