Files
HuggingFace_transformer/tests/models
Arthur e34da3ee3c [LlamaTokenizerFast] Refactor default llama (#28881)
* push legacy to fast as well

* super strange

* Update src/transformers/convert_slow_tokenizer.py

* make sure we are BC

* fix Llama test

* nit

* revert

* more test

* style

* update

* small update w.r.t tokenizers

* nit

* don't split

* lol

* add a test for `add_prefix_space=False`

* fix gemma tokenizer as well

* update

* fix gemma

* nicer failures

* fixup

* update

* fix the example for legacy = False

* use `huggyllama/llama-7b` for the PR doctest

* nit

* use from_slow

* fix llama
2024-04-23 23:12:59 +02:00
..
2024-04-05 17:49:46 +02:00
2024-04-01 07:51:00 +00:00
2024-04-18 15:18:52 +02:00
2024-04-22 17:11:13 +01:00
2024-04-01 07:51:00 +00:00
2023-08-02 20:22:36 +02:00
2024-04-15 17:03:03 +01:00
2023-08-02 20:22:36 +02:00
2024-04-17 17:59:07 +02:00
2023-09-05 10:12:25 +02:00
2023-09-05 10:12:25 +02:00
2024-04-01 07:51:00 +00:00
2024-04-19 18:31:43 +01:00
2023-08-02 20:22:36 +02:00
2024-01-11 16:18:27 +01:00
2024-03-20 15:51:12 +00:00
2024-01-11 16:18:27 +01:00
2022-05-03 14:42:02 +02:00