Nicolas Patry 68bb33d770 Fixing OPT fast tokenizer option. (#18753) ...

* Fixing OPT fast tokenizer option.

* Remove dependency on `pt`.

* Move it to GPT2 tokenization tests.

* Added a few tests.

2022-09-15 17:12:58 +02:00

..

__init__.py

Move test model folders (#17034)

2022-05-03 14:42:02 +02:00

test_modeling_flax_gpt2.py

Move test model folders (#17034)

2022-05-03 14:42:02 +02:00

test_modeling_gpt2.py

fix train_new_from_iterator in the case of byte-level tokenizers (#17549)

2022-06-08 15:30:41 +02:00

test_modeling_tf_gpt2.py

TF: TF 2.10 unpin + related onnx test skips (#18995)

2022-09-12 19:30:27 +01:00

test_tokenization_gpt2.py

Fixing OPT fast tokenizer option. (#18753)

2022-09-15 17:12:58 +02:00