SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity

[Docs] links to logits-processor-zoo (#35552)

Browse Source 
links to logits-processor-zoo
This commit is contained in:
Joao Gante
2025-01-08 13:36:30 +00:00
committed by GitHub
parent 3c1895aa65
commit 430d3d43a5
2 changed files with 9 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
docs/source/en/generation_strategies.md
View File

@@ -96,6 +96,12 @@ distribution over the entire vocabulary with various strategy-specific adjustmen
the decoding strategies that support multiple sequence candidates, e.g. variations of beam search and sampling. Decoding
strategies like greedy search and contrastive search return a single output sequence.
It is also possible to extend `generate()` with external libraries or handcrafted code. The `logits_processor` argument
allows you to pass custom [`LogitsProcessor`] instances, allowing you to manipulate the next token probability
distributions. Likewise, the `stopping_criteria` argument lets you set custom [`StoppingCriteria`] to stop text generation.
The [`logits-processor-zoo`](https://github.com/NVIDIA/logits-processor-zoo) library contains examples of external
`generate()`-compatible extensions.
## Save a custom decoding strategy with your model
If you would like to share your fine-tuned model with a specific generation configuration, you can:

5
docs/source/en/llm_tutorial.md
View File

@@ -265,8 +265,9 @@ While the autoregressive generation process is relatively straightforward, makin
### Related libraries
1. [`optimum`](https://github.com/huggingface/optimum), an extension of 🤗 Transformers that optimizes for specific hardware devices.
1. [`optimum`](https://github.com/huggingface/optimum), an extension of 🤗 Transformers that optimizes for specific hardware devices;
2. [`outlines`](https://github.com/outlines-dev/outlines), a library where you can constrain text generation (e.g. to generate JSON files);
3. [`SynCode`](https://github.com/uiuc-focal-lab/syncode), a library for context-free grammar guided generation. (e.g. JSON, SQL, Python)
3. [`SynCode`](https://github.com/uiuc-focal-lab/syncode), a library for context-free grammar guided generation (e.g. JSON, SQL, Python);
4. [`text-generation-inference`](https://github.com/huggingface/text-generation-inference), a production-ready server for LLMs;
5. [`text-generation-webui`](https://github.com/oobabooga/text-generation-webui), a UI for text generation;
6. [`logits-processor-zoo`](https://github.com/NVIDIA/logits-processor-zoo), containing additional options to control text generation with 🤗 Transformers. See our related [blog post](https://huggingface.co/blog/logits-processor-zoo).