Arthur 9ad815e412 [LlamaTokenizerFast] Adds edge cases for the template processor (#26606) ...

* make sure eos and bos are properly handled for fast tokenizer

* fix code llama as well

* nits

* fix the conversion script as well

* fix failing test

2023-10-06 16:40:54 +02:00

..

__init__.py

LLaMA Implementation (#21955)

2023-03-16 09:00:53 -04:00

test_modeling_llama.py

[core ] Integrate Flash attention 2 in most used models (#25598)

2023-09-22 17:42:10 +02:00

test_tokenization_llama.py

[LlamaTokenizerFast] Adds edge cases for the template processor (#26606)

2023-10-06 16:40:54 +02:00