Arthur 7cced021fa TF Sharded (#17713) ...

* initial commit

* update modeeling tf utils

* quality

* clean and update args

* update

* remove potential bug

* code quality

* update

* update max shard

* update tests for sharding from pretrained

* fix remaining test

* make style

* h5py if tf available

* update and fix test

* fix test

* style

* modified push to hub to support shard for TF

* quick fix

* update code

* merge branch main and style

* Apply suggestions from code review

Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

* update based on reviews

* update doc

* update and style

* Apply suggestions from code review

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update based on reviews

* fix typo

* style

Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

2022-06-21 18:01:08 +02:00

..

benchmark

[Test refactor 1/5] Per-folder tests reorganization (#15725)

2022-02-23 15:46:28 -05:00

deepspeed

deprecate is_torch_bf16_available (#17738)

2022-06-20 08:40:11 -04:00

extended

Update self-push workflow (#17177)

2022-05-13 16:28:00 +02:00

fixtures

add a warning in SpmConverter for sentencepiece's model using the byte fallback feature (#16629)

2022-04-11 11:06:10 +02:00

generation

[Generation Test] Make fast test actually fast (#17661)

2022-06-10 18:49:03 +02:00

models

Use 5e-5 For BigBird PT/Flax equivalence tests (#17780)

2022-06-21 17:55:26 +02:00

onnx

add onnx support for deberta and debertav2 (#17617)

2022-06-21 11:04:15 +02:00

optimization

[Test refactor 1/5] Per-folder tests reorganization (#15725)

2022-02-23 15:46:28 -05:00

pipelines

Add LongT5 model (#16792)

2022-06-13 22:36:58 +02:00

sagemaker

Black preview (#17217)

2022-05-12 16:25:55 -04:00

tokenization

fix train_new_from_iterator in the case of byte-level tokenizers (#17549)

2022-06-08 15:30:41 +02:00

trainer

Prepare transformers for v0.8.0 huggingface-hub release (#17716)

2022-06-21 11:51:18 -04:00

utils

🐛 Properly raise RepoNotFoundError when not authenticated (#17651)

2022-06-10 15:41:53 +02:00

__init__.py

GPU text generation: mMoved the encoded_prompt to correct device

2020-01-06 15:11:12 +01:00

test_configuration_common.py

Prepare transformers for v0.8.0 huggingface-hub release (#17716)

2022-06-21 11:51:18 -04:00

test_feature_extraction_common.py

Prepare transformers for v0.8.0 huggingface-hub release (#17716)

2022-06-21 11:51:18 -04:00

test_modeling_common.py

Use 5e-5 For BigBird PT/Flax equivalence tests (#17780)

2022-06-21 17:55:26 +02:00

test_modeling_flax_common.py

Use 5e-5 For BigBird PT/Flax equivalence tests (#17780)

2022-06-21 17:55:26 +02:00

test_modeling_tf_common.py

TF Sharded (#17713)

2022-06-21 18:01:08 +02:00

test_sequence_feature_extraction_common.py

Some tests misusing assertTrue for comparisons fix (#16771)

2022-04-19 14:44:08 +02:00

test_tokenization_common.py

Prepare transformers for v0.8.0 huggingface-hub release (#17716)

2022-06-21 11:51:18 -04:00