Raushan Turganbay
1806583390
[docs] Create page on inference servers with transformers backend ( #39550 )
...
* draft docs on inference servers
* Update docs/source/en/_toctree.yml
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
* update
* dic build failed
* Update docs/source/en/transformers_as_backend.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update docs/source/en/_toctree.yml
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update docs/source/en/transformers_as_backend.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update docs/source/en/transformers_as_backend.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update docs/source/en/transformers_as_backend.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update docs/source/en/transformers_as_backend.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update docs/source/en/transformers_as_backend.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update docs/source/en/transformers_as_backend.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update docs/source/en/transformers_as_backend.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update docs/source/en/transformers_as_backend.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update docs/source/en/transformers_as_backend.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update docs/source/en/transformers_as_backend.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update docs/source/en/transformers_as_backend.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update docs/source/en/transformers_as_backend.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update docs/source/en/transformers_as_backend.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update docs/source/en/transformers_as_backend.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update docs/source/en/transformers_as_backend.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update docs/source/en/transformers_as_backend.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update docs/source/en/transformers_as_backend.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update docs/source/en/transformers_as_backend.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update docs/source/en/transformers_as_backend.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update docs/source/en/transformers_as_backend.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update docs/source/en/transformers_as_backend.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update docs/source/en/transformers_as_backend.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Apply suggestions from code review
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* apply last suggestions
---------
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
2025-07-22 15:31:10 +02:00
Joao Gante
bf6c997685
[serve] Add speech to text (/v1/audio/transcriptions) ( #39434 )
...
* Scaffolding
* Explicit content
* Naïve Responses API streaming implementation
* Cleanup
* Scaffolding
* Explicit content
* Naïve Responses API streaming implementation
* Cleanup
* use openai
* validate request, including detecting unused fields
* dict indexing
* dict var access
* tmp commit (tests failing)
* add slow
* use oai output type in completions
* (little rebase errors)
* working spec?
* guard type hint
* type hints. fix state (CB can now load different models)
* type hints; fn names; error type
* add docstrings
* responses + kv cache
* metadata support; fix kv cache; error event
* add output_index and content_index
* docstrings
* add test_build_response_event
* docs/comments
* gate test requirements; terminate cb manager on model switch
* nasty type hints
* more type hints
* disable validation by default; enable force models
* todo
* experiment: base model from typed dict
* audio working
* fix bad rebase
* load audio with librosa
* implement timed models
* almost working
* make fixup
* fix tests
* transcription request type
* tokenizer -> processor
* add example in docs
---------
Co-authored-by: Lysandre <hi@lysand.re >
2025-07-17 14:29:57 +00:00
Lysandre Debut
de5ca373ac
Responses API in transformers serve ( #39155 )
...
* Scaffolding
* Explicit content
* Naïve Responses API streaming implementation
* Cleanup
* Responses API (to be merged into #39155 ) (#39338 )
* Scaffolding
* Explicit content
* Naïve Responses API streaming implementation
* Cleanup
* use openai
* validate request, including detecting unused fields
* dict indexing
* dict var access
* tmp commit (tests failing)
* add slow
* use oai output type in completions
* (little rebase errors)
* working spec?
* guard type hint
* type hints. fix state (CB can now load different models)
* type hints; fn names; error type
* add docstrings
* responses + kv cache
* metadata support; fix kv cache; error event
* add output_index and content_index
* docstrings
* add test_build_response_event
* docs/comments
* gate test requirements; terminate cb manager on model switch
* nasty type hints
* more type hints
* disable validation by default; enable force models
* todo
---------
Co-authored-by: Lysandre <hi@lysand.re >
* Slight bugfixes
* PR comments from #39338
* make fixup
---------
Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com >
Co-authored-by: Joao Gante <joao@huggingface.co >
2025-07-16 14:16:16 +02:00
Lucain
bf203aa9da
Update tiny-agents example ( #39245 )
2025-07-07 15:58:36 +02:00
Joao Gante
85d93cc6e3
[serve] Cursor support, move docs into separate page, add more examples ( #39133 )
...
* jan docs
* rm
* [cursor] tmp commit
* Cursor working :D
* Update docs/source/en/serving.md
Co-authored-by: Pedro Cuenca <pedro@huggingface.co >
* Update docs/source/en/serving.md
Co-authored-by: Pedro Cuenca <pedro@huggingface.co >
* Update docs/source/en/serving.md
Co-authored-by: Pedro Cuenca <pedro@huggingface.co >
* Update docs/source/en/serving.md
Co-authored-by: Pedro Cuenca <pedro@huggingface.co >
* Update docs/source/en/serving.md
Co-authored-by: Pedro Cuenca <pedro@huggingface.co >
* Update docs/source/en/serving.md
Co-authored-by: Pedro Cuenca <pedro@huggingface.co >
* Update docs/source/en/serving.md
Co-authored-by: Pedro Cuenca <pedro@huggingface.co >
* Update docs/source/en/serving.md
Co-authored-by: Pedro Cuenca <pedro@huggingface.co >
* Update src/transformers/commands/serving.py
Co-authored-by: Pedro Cuenca <pedro@huggingface.co >
* cursor docs
* try to fix agents/tools docs?
* try to fix agents/tools docs?
* Update docs/source/en/serving.md
Co-authored-by: Pedro Cuenca <pedro@huggingface.co >
* add transformers chat example with transformers serve
---------
Co-authored-by: Pedro Cuenca <pedro@huggingface.co >
2025-07-03 17:04:16 +01:00
湛露先生
cc68070d41
fix docs serving typos. ( #37936 )
...
Signed-off-by: zhanluxianshen <zhanluxianshen@163.com >
2025-05-06 14:32:44 +01:00
Steven Liu
e9756cdbc7
[docs] Serving LLMs ( #36522 )
...
* initial
* fix
* model-impl
2025-03-10 13:14:19 -07:00
