SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity

[Docs] links to logits-processor-zoo (#35552)

Browse Source 
links to logits-processor-zoo
This commit is contained in:
Joao Gante
2025-01-08 13:36:30 +00:00
committed by GitHub
parent 3c1895aa65
commit 430d3d43a5
2 changed files with 9 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
docs/source/en/generation_strategies.md
View File

@@ -96,6 +96,12 @@ distribution over the entire vocabulary with various strategy-specific adjustmen
the decoding strategies that support multiple sequence candidates, e.g. variations of beam search and sampling. Decoding the decoding strategies that support multiple sequence candidates, e.g. variations of beam search and sampling. Decoding
strategies like greedy search and contrastive search return a single output sequence. strategies like greedy search and contrastive search return a single output sequence.
It is also possible to extend `generate()` with external libraries or handcrafted code. The `logits_processor` argument
allows you to pass custom [`LogitsProcessor`] instances, allowing you to manipulate the next token probability
distributions. Likewise, the `stopping_criteria` argument lets you set custom [`StoppingCriteria`] to stop text generation.
The [`logits-processor-zoo`](https://github.com/NVIDIA/logits-processor-zoo) library contains examples of external
`generate()`-compatible extensions.
## Save a custom decoding strategy with your model ## Save a custom decoding strategy with your model
If you would like to share your fine-tuned model with a specific generation configuration, you can: If you would like to share your fine-tuned model with a specific generation configuration, you can:

5
docs/source/en/llm_tutorial.md
View File

@@ -265,8 +265,9 @@ While the autoregressive generation process is relatively straightforward, makin
### Related libraries ### Related libraries
1. [`optimum`](https://github.com/huggingface/optimum), an extension of 🤗 Transformers that optimizes for specific hardware devices. 1. [`optimum`](https://github.com/huggingface/optimum), an extension of 🤗 Transformers that optimizes for specific hardware devices;
2. [`outlines`](https://github.com/outlines-dev/outlines), a library where you can constrain text generation (e.g. to generate JSON files); 2. [`outlines`](https://github.com/outlines-dev/outlines), a library where you can constrain text generation (e.g. to generate JSON files);
3. [`SynCode`](https://github.com/uiuc-focal-lab/syncode), a library for context-free grammar guided generation. (e.g. JSON, SQL, Python) 3. [`SynCode`](https://github.com/uiuc-focal-lab/syncode), a library for context-free grammar guided generation (e.g. JSON, SQL, Python);
4. [`text-generation-inference`](https://github.com/huggingface/text-generation-inference), a production-ready server for LLMs; 4. [`text-generation-inference`](https://github.com/huggingface/text-generation-inference), a production-ready server for LLMs;
5. [`text-generation-webui`](https://github.com/oobabooga/text-generation-webui), a UI for text generation; 5. [`text-generation-webui`](https://github.com/oobabooga/text-generation-webui), a UI for text generation;
6. [`logits-processor-zoo`](https://github.com/NVIDIA/logits-processor-zoo), containing additional options to control text generation with 🤗 Transformers. See our related [blog post](https://huggingface.co/blog/logits-processor-zoo).