[RoPE] abstract dynamic RoPE update under a decorator (#37249)

* dynamic rope decorator

* longrope; shorter fwd pass

* propper docstring

* make fixup
This commit is contained in:
Joao Gante
2025-04-04 14:27:28 +01:00
committed by GitHub
parent 3d40bda30e
commit ad3d157188
52 changed files with 527 additions and 1833 deletions

View File

@@ -25,6 +25,10 @@ Most of those are only useful if you are studying the code of the models in the
[[autodoc]] AttentionInterface
- register
## Rotary Position Embedding Functions
[[autodoc]] dynamic_rope_update
## Pytorch custom modules
[[autodoc]] pytorch_utils.Conv1D