From 430d3d43a56b72be4213ba3e2553cdb34ea0b471 Mon Sep 17 00:00:00 2001 From: Joao Gante Date: Wed, 8 Jan 2025 13:36:30 +0000 Subject: [PATCH] [Docs] links to `logits-processor-zoo` (#35552) links to logits-processor-zoo --- docs/source/en/generation_strategies.md | 6 ++++++ docs/source/en/llm_tutorial.md | 5 +++-- 2 files changed, 9 insertions(+), 2 deletions(-) diff --git a/docs/source/en/generation_strategies.md b/docs/source/en/generation_strategies.md index 47032a2a29..757f42cdf5 100644 --- a/docs/source/en/generation_strategies.md +++ b/docs/source/en/generation_strategies.md @@ -96,6 +96,12 @@ distribution over the entire vocabulary with various strategy-specific adjustmen the decoding strategies that support multiple sequence candidates, e.g. variations of beam search and sampling. Decoding strategies like greedy search and contrastive search return a single output sequence. +It is also possible to extend `generate()` with external libraries or handcrafted code. The `logits_processor` argument +allows you to pass custom [`LogitsProcessor`] instances, allowing you to manipulate the next token probability +distributions. Likewise, the `stopping_criteria` argument lets you set custom [`StoppingCriteria`] to stop text generation. +The [`logits-processor-zoo`](https://github.com/NVIDIA/logits-processor-zoo) library contains examples of external +`generate()`-compatible extensions. + ## Save a custom decoding strategy with your model If you would like to share your fine-tuned model with a specific generation configuration, you can: diff --git a/docs/source/en/llm_tutorial.md b/docs/source/en/llm_tutorial.md index 097d7bf1e9..eaf59c741c 100644 --- a/docs/source/en/llm_tutorial.md +++ b/docs/source/en/llm_tutorial.md @@ -265,8 +265,9 @@ While the autoregressive generation process is relatively straightforward, makin ### Related libraries -1. [`optimum`](https://github.com/huggingface/optimum), an extension of 🤗 Transformers that optimizes for specific hardware devices. +1. [`optimum`](https://github.com/huggingface/optimum), an extension of 🤗 Transformers that optimizes for specific hardware devices; 2. [`outlines`](https://github.com/outlines-dev/outlines), a library where you can constrain text generation (e.g. to generate JSON files); -3. [`SynCode`](https://github.com/uiuc-focal-lab/syncode), a library for context-free grammar guided generation. (e.g. JSON, SQL, Python) +3. [`SynCode`](https://github.com/uiuc-focal-lab/syncode), a library for context-free grammar guided generation (e.g. JSON, SQL, Python); 4. [`text-generation-inference`](https://github.com/huggingface/text-generation-inference), a production-ready server for LLMs; 5. [`text-generation-webui`](https://github.com/oobabooga/text-generation-webui), a UI for text generation; +6. [`logits-processor-zoo`](https://github.com/NVIDIA/logits-processor-zoo), containing additional options to control text generation with 🤗 Transformers. See our related [blog post](https://huggingface.co/blog/logits-processor-zoo).