Add Space to Bitsandbytes doc (#36834)

* add space

* address review
This commit is contained in:
Mohamed Mekkouri
2025-03-19 18:56:07 +01:00
committed by GitHub
parent f39f4960f3
commit 258dd9cc69
2 changed files with 14 additions and 2 deletions

View File

@@ -20,7 +20,10 @@ rendered properly in your Markdown viewer.
[LLM.int8()](https://hf.co/papers/2208.07339) is a quantization method that aims to make large language model inference more accessible without significant degradation. Unlike naive 8-bit quantization, which can result in loss of critical information and accuracy, LLM.int8() dynamically adapts to ensure sensitive components of the computation retain higher precision when needed.
QLoRA, or 4-bit quantization, compresses a model even further to 4-bits and inserts a small set of trainable low-rank adaptation (LoRA) weights to allowing training.
QLoRA, or 4-bit quantization, compresses a model even further to 4-bits and inserts a small set of trainable low-rank adaptation (LoRA) weights to allowing training.
> **Note:** For a user-friendly quantization experience, you can use the `bitsandbytes` [community space](https://huggingface.co/spaces/bnb-community/bnb-my-repo).
Run the command below to install bitsandbytes.