SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity

MinNewTokensLengthLogitsProcessor for .generate method #20814 (#20892)

Browse Source 
* feat: add min new length logit processor

* test: add min new length logit processor

* docs: add MinNewTokensLengthLogitsProcessor

* feat: import MinNewTokensLengthLogitsProcessor

* fix: update pytorch dummy objects

* refactor & fix: rename attributes and var and get rid of dynamic attribute

* tests: align test with new interface

* docs: fix typo

* docs: minor clarification

* Empty-Commit

* empty commit

* run automated quality edits

Co-authored-by: Joao Gante <joao@huggingface.co>
This commit is contained in:
Konstantin Kotik
2023-01-03 14:29:02 +03:00
committed by GitHub
parent 4fd89e4978
commit 367fdf3330
6 changed files with 99 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
docs/source/en/internal/generation_utils.mdx
View File

@@ -116,6 +116,9 @@ generation.
[[autodoc]] MinLengthLogitsProcessor
- __call__
[[autodoc]] MinNewTokensLengthLogitsProcessor
- __call__
[[autodoc]] TemperatureLogitsWarper
- __call__