Tanay Mehta 12f043eaea Fix MarianTokenizer to remove metaspace character in decode (#26091) ...

* add: check to remove metaspace from marian tokenizer

* fix: metaspace character being removed from everywhere

* fix: remove redundant check at top

* add: test for marian tokenizer decode fix

* fix: simplified the test

2023-09-12 21:53:31 +02:00

..

benchmark

[Test refactor 1/5] Per-folder tests reorganization (#15725)

2022-02-23 15:46:28 -05:00

bettertransformer

Add methods to PreTrainedModel to use PyTorch's BetterTransformer (#21259)

2023-04-27 11:03:42 +02:00

deepspeed

deepspeed resume from ckpt fixes and adding support for deepspeed optimizer and HF scheduler (#25863)

2023-09-05 22:31:20 +05:30

extended

[tests] switch to torchrun (#22712)

2023-04-12 08:25:45 -07:00

fixtures

[WIP] add SpeechT5 model (#18922)

2023-02-03 12:43:46 -05:00

generation

Fix ExponentialDecayLengthPenalty negative logits issue (#25594)

2023-09-12 12:50:41 +01:00

models

Fix MarianTokenizer to remove metaspace character in decode (#26091)

2023-09-12 21:53:31 +02:00

optimization

Make schedulers picklable by making lr_lambda fns global (#21768)

2023-03-02 12:08:43 -05:00

peft_integration

[PEFT] Fix PeftConfig save pretrained when calling add_adapter (#25738)

2023-08-25 08:19:11 +02:00

pipelines

[CI] Fix red CI and ERROR failed should show (#25995)

2023-09-05 20:16:00 +02:00

quantization

modify context length for GPTQ + version bump (#25899)

2023-09-06 11:45:47 -04:00

repo_utils

Document check copies (#25291)

2023-08-04 14:56:29 +02:00

sagemaker

Avoid invalid escape sequences, use raw strings (#22936)

2023-04-25 09:17:56 -04:00

tokenization

[ PreTrainedTokenizerFast] Keep properties from fast tokenizer (#25053)

2023-07-25 18:45:01 +02:00

tools

Add support for for loops in python interpreter (#24429)

2023-06-26 09:58:14 -04:00

trainer

enable optuna multi-objectives feature (#25969)

2023-09-12 18:01:22 +01:00

utils

Fix test_load_img_url_timeout (#25976)

2023-09-05 11:34:28 +02:00

__init__.py

GPU text generation: mMoved the encoded_prompt to correct device

2020-01-06 15:11:12 +01:00

test_backbone_common.py

Add ViTDet (#25524)

2023-08-29 10:03:52 +01:00

test_configuration_common.py

Deal with nested configs better in base class (#25237)

2023-08-04 14:56:09 +02:00

test_configuration_utils.py

Deal with nested configs better in base class (#25237)

2023-08-04 14:56:09 +02:00

test_feature_extraction_common.py

Split common test from core tests (#24284)

2023-06-15 07:30:24 -04:00

test_feature_extraction_utils.py

Split common test from core tests (#24284)

2023-06-15 07:30:24 -04:00

test_image_processing_common.py

Input data format (#25464)

2023-08-16 17:45:02 +01:00

test_image_processing_utils.py

Run hub tests (#24807)

2023-07-13 15:25:45 -04:00

test_image_transforms.py

Add input_data_format argument, image transforms (#25462)

2023-08-11 15:09:31 +01:00

test_modeling_common.py

[resize_embedding] Introduce pad_to_multiple_of and guidance (#25088)

2023-08-17 17:00:32 +02:00

test_modeling_flax_common.py

Split common test from core tests (#24284)

2023-06-15 07:30:24 -04:00

test_modeling_flax_utils.py

Split common test from core tests (#24284)

2023-06-15 07:30:24 -04:00

test_modeling_tf_common.py

Skip test_onnx_runtime_optimize for now (#25560)

2023-08-17 11:23:16 +02:00

test_modeling_tf_utils.py

Split common test from core tests (#24284)

2023-06-15 07:30:24 -04:00

test_modeling_utils.py

Skip warning if tracing with dynamo (#25581)

2023-09-08 21:13:33 +02:00

test_pipeline_mixin.py

Add Text-To-Speech pipeline (#24952)

2023-08-17 17:34:47 +01:00

test_sequence_feature_extraction_common.py

Fix typo (#25966)

2023-09-05 10:12:25 +02:00

test_tokenization_common.py

[split_special_tokens] Add support for split_special_tokens argument to encode (#25081)

2023-08-18 13:26:27 +02:00

test_tokenization_utils.py

Split common test from core tests (#24284)

2023-06-15 07:30:24 -04:00