Cache: Static cache as a standalone object (#30476)

2024-04-30 16:37:19 +01:00
parent 0ae789e043
commit 75bbfd5b22
20 changed files with 377 additions and 424 deletions
--- a/docs/source/en/internal/generation_utils.md
+++ b/docs/source/en/internal/generation_utils.md
@@ -362,3 +362,4 @@ A [`Constraint`] can be used to force the generation to include specific tokens
 [[autodoc]] StaticCache
    - update
    - get_seq_length
+    - reorder_cache