From 258dd9cc693d1ad2c43e255cde73ce500d30b303 Mon Sep 17 00:00:00 2001 From: Mohamed Mekkouri <93391238+MekkCyber@users.noreply.github.com> Date: Wed, 19 Mar 2025 18:56:07 +0100 Subject: [PATCH] Add Space to Bitsandbytes doc (#36834) * add space * address review --- docs/source/en/quantization/bitsandbytes.md | 5 ++++- docs/source/en/quantization/overview.md | 11 ++++++++++- 2 files changed, 14 insertions(+), 2 deletions(-) diff --git a/docs/source/en/quantization/bitsandbytes.md b/docs/source/en/quantization/bitsandbytes.md index 2f87cb0dcd..e9b581b89f 100644 --- a/docs/source/en/quantization/bitsandbytes.md +++ b/docs/source/en/quantization/bitsandbytes.md @@ -20,7 +20,10 @@ rendered properly in your Markdown viewer. [LLM.int8()](https://hf.co/papers/2208.07339) is a quantization method that aims to make large language model inference more accessible without significant degradation. Unlike naive 8-bit quantization, which can result in loss of critical information and accuracy, LLM.int8() dynamically adapts to ensure sensitive components of the computation retain higher precision when needed. -QLoRA, or 4-bit quantization, compresses a model even further to 4-bits and inserts a small set of trainable low-rank adaptation (LoRA) weights to allowing training. +QLoRA, or 4-bit quantization, compresses a model even further to 4-bits and inserts a small set of trainable low-rank adaptation (LoRA) weights to allowing training. + +> **Note:** For a user-friendly quantization experience, you can use the `bitsandbytes` [community space](https://huggingface.co/spaces/bnb-community/bnb-my-repo). + Run the command below to install bitsandbytes. diff --git a/docs/source/en/quantization/overview.md b/docs/source/en/quantization/overview.md index 1c0f48988d..d69675e909 100644 --- a/docs/source/en/quantization/overview.md +++ b/docs/source/en/quantization/overview.md @@ -46,4 +46,13 @@ Use the Space below to help you pick a quantization method depending on your har If you are new to quantization, we recommend checking out these beginner-friendly quantization courses in collaboration with DeepLearning.AI. * [Quantization Fundamentals with Hugging Face](https://www.deeplearning.ai/short-courses/quantization-fundamentals-with-hugging-face/) -* [Quantization in Depth](https://www.deeplearning.ai/short-courses/quantization-in-depth \ No newline at end of file +* [Quantization in Depth](https://www.deeplearning.ai/short-courses/quantization-in-depth) + +## User-Friendly Quantization Tools + +If you are looking for a user-friendly quantization experience, you can use the following community spaces and notebooks: + +* [Bitsandbytes Space](https://huggingface.co/spaces/bnb-community/bnb-my-repo) +* [GGUF Space](https://huggingface.co/spaces/ggml-org/gguf-my-repo) +* [MLX Space](https://huggingface.co/spaces/mlx-community/mlx-my-repo) +* [AuoQuant Notebook](https://colab.research.google.com/drive/1b6nqC7UZVt8bx4MksX7s656GXPM-eWw4?usp=sharing#scrollTo=ZC9Nsr9u5WhN) \ No newline at end of file