Nicolas Patry 80d712fac6 Adding new argument max_new_tokens for generate. (#11476) ...

* Adding new argument `max_new_tokens` for generate.

This is a proposal to add a new argument `max_new_tokens` to `generate`.
This include a `MaxNewTokensCriteria` that enables callers that don't
know about the token length ahead (like pipelines callers) to manage
more easily the length of their generated output.

* Adding a test for the user warning when both`max_length` and
`max_new_tokens` are used together.

* Removed redundant `no_grad`.

2021-05-27 14:22:58 +02:00

69 KiB

Raw Blame History

View Raw