Generate: basic token streaming (#22449)

* haha tokens go brrrr
2023-03-30 12:00:12 +01:00
parent f0aeb1be17
commit 228792a9dc
8 changed files with 230 additions and 7 deletions
--- a/docs/source/en/internal/generation_utils.mdx
+++ b/docs/source/en/internal/generation_utils.mdx
@@ -265,3 +265,7 @@ A [`Constraint`] can be used to force the generation to include specific tokens
 [[autodoc]] top_k_top_p_filtering

 [[autodoc]] tf_top_k_top_p_filtering
+
+## Streamers
+
+[[autodoc]] TextStreamer