Nicolas Patry 68bb33d770 Fixing OPT fast tokenizer option. (#18753) ...

* Fixing OPT fast tokenizer option.

* Remove dependency on `pt`.

* Move it to GPT2 tokenization tests.

* Added a few tests.

2022-09-15 17:12:58 +02:00

..

benchmark

[Test refactor 1/5] Per-folder tests reorganization (#15725)

2022-02-23 15:46:28 -05:00

deepspeed

deprecate is_torch_bf16_available (#17738)

2022-06-20 08:40:11 -04:00

extended

Update self-push workflow (#17177)

2022-05-13 16:28:00 +02:00

fixtures

add a warning in SpmConverter for sentencepiece's model using the byte fallback feature (#16629)

2022-04-11 11:06:10 +02:00

generation

🚨🚨🚨 Optimize Top P Sampler and fix edge case (#18984)

2022-09-15 15:50:11 +02:00

mixed_int8

[bnb] Minor modifications (#18631)

2022-08-17 00:48:10 +02:00

models

Fixing OPT fast tokenizer option. (#18753)

2022-09-15 17:12:58 +02:00

onnx

Add SegFormer ONNX support (#18006)

2022-08-31 20:58:44 +02:00

optimization

[Test refactor 1/5] Per-folder tests reorganization (#15725)

2022-02-23 15:46:28 -05:00

pipelines

Update image segmentation pipeline test (#18731)

2022-09-15 07:32:31 -04:00

sagemaker

transformers-cli login => huggingface-cli login (#18490)

2022-08-06 09:42:55 +02:00

tokenization

fix train_new_from_iterator in the case of byte-level tokenizers (#17549)

2022-06-08 15:30:41 +02:00

trainer

small change (#18584)

2022-08-12 20:04:38 +02:00

utils

Clean up utils.hub using the latest from hf_hub (#18857)

2022-09-02 10:30:06 -04:00

__init__.py

GPU text generation: mMoved the encoded_prompt to correct device

2020-01-06 15:11:12 +01:00

test_configuration_common.py

Fix mock in test_cached_files_are_used_when_internet_is_down (#18804)

2022-08-29 15:56:08 +02:00

test_feature_extraction_common.py

Fix mock in test_cached_files_are_used_when_internet_is_down (#18804)

2022-08-29 15:56:08 +02:00

test_modeling_common.py

Add DocumentQuestionAnswering pipeline (#18414)

2022-09-07 13:38:49 -04:00

test_modeling_flax_common.py

[JAX] Replace all jax.tree_* calls with jax.tree_util.tree_* (#18361)

2022-09-09 15:18:56 +02:00

test_modeling_tf_common.py

Update serving signatures and make sure we actually use them (#19034)

2022-09-15 14:34:22 +01:00

test_sequence_feature_extraction_common.py

Some tests misusing assertTrue for comparisons fix (#16771)

2022-04-19 14:44:08 +02:00

test_tokenization_common.py

Fix mock in test_cached_files_are_used_when_internet_is_down (#18804)

2022-08-29 15:56:08 +02:00