Arthur 5b5ee235f3 [LlamaTokenizer] tokenize nits. (#25793) ...

* return when length is zero

* Add tests

Co-authored-by:  Avnish Narayan <38871737avnishn@users.noreply.github.com>

* Co-authored-by: avnishn
<38871737+avnishn@users.noreply.github.com>

* codeLlama doc should not be on Main

* update test

---------

Co-authored-by: Avnish Narayan <38871737avnishn@users.noreply.github.com>

2023-08-29 15:08:14 +02:00

..

__init__.py

Move test model folders (#17034)

2022-05-03 14:42:02 +02:00

test_modeling_flax_t5.py

CI with num_hidden_layers=2 🚀🚀🚀 (#25266)

2023-08-02 20:22:36 +02:00

test_modeling_t5.py

CI with num_hidden_layers=2 🚀🚀🚀 (#25266)

2023-08-02 20:22:36 +02:00

test_modeling_tf_t5.py

Skip test_beam_search_xla_generate_simple for T5 (#25566)

2023-08-17 15:30:46 +02:00

test_tokenization_t5.py

[LlamaTokenizer] tokenize nits. (#25793)

2023-08-29 15:08:14 +02:00