Files
HuggingFace_transformer/docs/source
Arthur 88832c01c8 [Whisper] Add conversion script for the tokenizer (#27338)
* draft

* updates

* full conversion taken from `https://gist.github.com/xenova/a452a6474428de0182b17605a98631ee`

* psuh

* nits

* updates

* more nits

* Add co author

Co-authored-by: Joshua Lochner <admin@xenova.com>

* fixup

* cleanup

* styling

* add proper path

* update

* nits

* don't  push the exit

* clean

* update whisper doc

* don't error out if tiktoken is not here

* make sure we are BC with conversion

* nit

* Update docs/source/en/model_doc/whisper.md

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

* merge and update

* update markdwon

* Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>

---------

Co-authored-by: Joshua Lochner <admin@xenova.com>
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
2023-11-07 15:07:55 +01:00
..
2023-10-02 09:56:40 -07:00
2023-11-06 19:45:03 +00:00
2023-09-04 11:15:12 +01:00
2023-11-06 19:45:03 +00:00
2023-11-06 19:45:03 +00:00
2023-11-06 19:45:03 +00:00
2023-11-06 19:45:03 +00:00
2023-11-06 19:45:03 +00:00