SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity
8,360 Commits 5 Branches 239 Tags
3772af49ceba348f2c9c5bbbb7f7c12e35d2a6eb
Commit Graph

8360 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
NielsRogge
3772af49ce [Tests] Improve vision tests (#14458) 
* Improve tests

* Install vision for tf tests
 2021-11-24 15:22:20 +01:00
Lysandre Debut
f2e90bcb8f Fix feature extraction utils import (#14515) 2021-11-24 09:03:21 -05:00
Vladimir Maryasin
6c4d688ffa add cache_dir for tokenizer verification loading (#14508) 
When loading a pretrained tokenizer, a verification is done to ensure
that the actual tokenizer class matches the class it was called from.
If the tokenizer is absent, its config file is loaded from the repo.

However, the cache_dir for downloading is not provided, which leads to
ignoring of the user-specified cache_dir, storing files in several
places and and may result in incorrect warnings when the default
cache_dir is unreachsble.

This commit fixes that.
 2021-11-24 06:22:03 -05:00
Stas Bekman
956a483173 [deepspeed] zero inference (#14253) 
* [deepspeed] zero inference

* only z3 makes sense for inference

* fix and style

* docs

* rework

* fix test

* Apply suggestions from code review

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* responding to suggestions

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2021-11-23 14:09:15 -08:00
Nicholas Broad
69e16abf98 Switch from using sum for flattening lists of lists in group_texts (#14472) 
* remove sum for list flattening

* change to chain(*)

* make chain object a list

* delete empty lines

per sgugger's suggestions

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

Co-authored-by: Nicholas Broad <nicholas@nmbroad.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2021-11-22 16:17:26 -05:00
Valentin
0b7d053c13 fixes some key names for in LayoutLMv2 / LayoutXLM tokenizers (#14493) 
in case of left padding_side there was a copy/paste error
assigning the bbox data to the labels
 2021-11-22 16:00:43 -05:00
Sylvain Gugger
204d251310 Auto processor (#14465) 
* Add AutoProcessor class

* Init and tests

* Add doc

* Fix init

* Update src/transformers/models/auto/processing_auto.py

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

* Reverts to tokenizer or feature extractor when available

* Adapt test

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
 2021-11-22 12:17:38 -05:00
Stas Bekman
11f65d4158 [test] add test for --config_overrides (#14466) 
* add test for --config_overrides

* remove unneeded parts of the test
 2021-11-22 11:33:43 -05:00
Daniel Stancl
e0e2da1194 Improve a add-new-pipeline docs a bit (#14485) 2021-11-22 10:35:49 -05:00
Nicolas Patry
a4553e6c64 Moving pipeline tests from Narsil to hf-internal-testing. (#14463) 
* Moving everything to `hf-internal-testing`.

* Fixing test values.

* Moving to other repo.

* Last touch?
 2021-11-22 04:40:45 -05:00
Sylvain Gugger
1a92bc5788 Fix dummy objects for quantization (#14478) 
* Fix dummy objects for quantization

* Add more models
 2021-11-21 17:39:20 -05:00
Alexander Measure
c9d2cf855a add Tuple as possible type hint for EvalPredictions label_ids (#14473) 
* Update trainer_utils.py

* add Tuple type hints to all label_ids outputs

affects EvalLoopOutput and PredicctionOutput
 2021-11-21 10:31:09 -05:00
Shang Zhang
a59e7c1ed4 Add QDQBert model and quantization examples of SQUAD task (#14066) 
* clean up branch for add-qdqbert-model

* README update for QAT example; update docstrings in modeling_qdqbert.py

* Update qdqbert.rst

* Update README.md

* Update README.md

* calibration data using traning set; QAT example runs in fp32

* re-use BERTtokenizer for qdqbert

* Update docs/source/model_doc/qdqbert.rst

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update docs/source/model_doc/qdqbert.rst

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update docs/source/model_doc/qdqbert.rst

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* remove qdqbert tokenizer

* Update qdqbert.rst

* update evaluate-hf-trt-qa.py

* update configuration_qdqbert.py

* update modeling_qdqbert.py: add copied statement; replace assert with ValueError

* update copied from statement

* add is_quantization_available; run make fix-copies

* unittest add require_quantization

* add backend dependency to qdqbert model

* update README; update evaluate script; make style

* lint

* docs qdqbert update

* circleci build_doc add pytorch-quantization for qdqbert

* update README

* update example readme with instructions to upgrade TensorRT to 8.2

* Update src/transformers/models/qdqbert/configuration_qdqbert.py

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

* Update src/transformers/models/qdqbert/configuration_qdqbert.py

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

* Update src/transformers/models/qdqbert/configuration_qdqbert.py

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

* Update src/transformers/models/qdqbert/configuration_qdqbert.py

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

* change quantization to pytorch_quantization for backend requirement

* feed_forward_chunking not supported in QDQBert

* make style

* update model docstrings and comments in testing scripts

* rename example to quantization-qdqbert; rename example scripts from qat to quant

* Update src/transformers/models/qdqbert/modeling_qdqbert.py

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

* rm experimental functions in quant_trainer

* qa cleanup

* make fix-copies for docs index.rst

* fix doctree; use post_init() for qdqbert

* fix early device assignment for qdqbert

* fix CI:Model templates runner

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
 2021-11-19 13:33:39 -05:00
Nicolas Patry
81fe8afaac Adding support for hidden_states and attentions in unbatching (#14420) 
support.
 2021-11-19 15:37:52 +01:00
Patrick von Platen
f25a9332e8 [Generation] Allow inputs_embeds as an input (#14443) 
* up

* finalize

* finalize

* finish

* Update src/transformers/generation_utils.py

* apply feedback
 2021-11-19 15:35:06 +01:00
NielsRogge
0490b98877 [ImageGPT] Small fixes (#14460) 
* Add integration test

* Fix typo
 2021-11-19 15:15:02 +01:00
Lysandre Debut
331c3d2aa0 Add GitPython to quality tools (#14459) 
* Update setup.py

* Update setup.py

* Update setup.py

* Remove GitPython install
 2021-11-19 08:43:48 -05:00
Patrick von Platen
efea0f868b [Speech Recognition] More examples 
Add more XLS-R training runs to the official examples
 2021-11-18 23:42:02 +01:00
Stas Bekman
72a6bf33c0 [Bert, et al] fix early device assignment (#14447) 
* fix early device assignment

* more models
 2021-11-18 11:47:49 -08:00
Sylvain Gugger
83ef8bcac2 Fix finite IterableDataset test on multiple GPUs (#14445) 2021-11-18 10:25:06 -05:00
NielsRogge
da36c557f7 Add ImageGPT (#14240) 
* First draft

* More improvements

* Improve conversion script

* Fix init weights for layer norm

* Fix correct model for conversion script

* Don't tie input and output embeddings

* Add print statements for debugging

* Add print statements for debugging

* Fix vocab size of model

* Improve documentation, remove fast tokenizer

* Add ImageGPTForImageClassification, improve docs

* Fix docs issue

* Set verbosity level back to info

* Improve tests

* Fix tests and add figure

* Delete tokenizer file

* Remove ImageGPTTokenizer from init files

* Remove ImageGPTLayer from init files

* Remove ImageGPT tokenizer from docs

* First draft of ImageGPTFeatureExtractor

* Fix typo

* Fix bug

* More improvements

* Apply suggestions from code review, add tests for feature extractor

* Fix layernorm

* Update save_pretrained method

* Fix issue

* Make all tests of ImageGPTFeatureExtractor pass

* Update code examples

* Rename model inputs to pixel_values

* Improve code examples

* Update init_weights to post_init

* Fix post_init
 2021-11-18 16:24:34 +01:00
Sylvain Gugger
d83b0e0c07 Add a post init method to all models (#14431) 
* Add a post init method to all models

* Fix tests

* Fix last tests

* Fix templates

* Add comment

* Forgot to save
 2021-11-18 08:38:09 -05:00
NielsRogge
08816de16a Fix code example (#14441) 2021-11-18 11:26:54 +01:00
William Held
01f8e639d3 Recover Deleted XNLI Instructions (#14437) 2021-11-17 20:16:47 -05:00
N
1991da07f7 [WIP] Ensure TF model configs can be converted to proper JSON (#14415) 
* test: make sure model configs are jsonifiable

* fix: return python dict instead of config object

* fix: accept pretrained config and use correct class

* Re-enabling slow tests and applying them to core models only

* Re-enabling slow tests and applying them to core models only

* Add new test file to fetcher

* Remove tooslow tests from test_modeling_tf_common.py

* make style

* Style fixes

* Style fixes

* Style fixes

* Style fixes

* Adding core tests to GPT2 and BART

* Removing unused imports

Co-authored-by: niklas.fruehauf <niklas.fruehauf@sovanta.com>
Co-authored-by: matt <rocketknight1@gmail.com>
 2021-11-17 20:24:39 +00:00
Patrick von Platen
754202de4f [Bart] Fix docs (#14434) 2021-11-17 19:02:33 +01:00
Antonio Carlos Falcão Petri
7544efc92e [Gradient checkpoining] Update Wav2Vec scripts (#14036) 
Co-authored-by: Stas Bekman <stas@stason.org>
 2021-11-17 18:37:21 +01:00
Lysandre
c6c075544d Docs for version v4.12.5 2021-11-17 11:39:12 -05:00
NielsRogge
a2864a50e7 Improve semantic segmentation models (#14355) 
* Improve tests

* Improve documentation

* Add ignore_index attribute

* Add semantic_ignore_index to BEiT model

* Add segmentation maps argument to BEiTFeatureExtractor

* Simplify SegformerFeatureExtractor and corresponding tests

* Improve tests

* Apply suggestions from code review

* Minor docs improvements

* Streamline segmentation map tests of SegFormer and BEiT

* Improve reduce_labels docs and test

* Fix code quality

* Fix code quality again
 2021-11-17 15:29:58 +01:00
Patrick von Platen
700a748fe6 [Wav2Vec2] Add New Wav2Vec2 Translation (#14392) 
* add new wav2vec2 translation

* correct

* up

* add tests

* correct end copy

* correct more

* up

* correct unispeech sat

* finish

* finalize

* finish

* up
 2021-11-17 14:38:56 +01:00
Sylvain Gugger
b567510cff Debug doc (#14424) 
* Create branch for tests

* Pin first upgrade

* Really pin

* Polish fix
 2021-11-16 18:58:07 -05:00
Lysandre
888fb21159 Docs for v4.12.4 2021-11-16 17:40:58 -05:00
Valentin
a33168aa78 Avoid looping when data exhausted (#14413) 
* stop training when a finite IterableDataset is exhausted

when using an iterable dataset num_epochs is set to
sys.maxsize to make sure all data is consumed
likewise we want to set max_steps high enough
but still stop when all data is consumed

(cherry picked from commit 6f0e1d6363153da9051e93acffe1cbab3a3f3b12)

* fix typo flase -> false

* add test for stopping training on exhausted finite iterable dataset

* remove redundant gradient_accumulation_steps

* run make style

reformat training_args docstring
 2021-11-16 16:50:04 -05:00
Sylvain Gugger
3e8d17e66d Add forward method to dummy models (#14419) 
* Add forward method to dummy models

* Fix quality
 2021-11-16 09:24:40 -05:00
Sylvain Gugger
040fd47162 Fix gradient_checkpointing backward compatibility (#14408) 
* Fix gradient_checkpointing backward compatibility

* Remove needless line

* make sure mask prob is big enough and length small enough

* Fix tests

Co-authored-by: patrickvonplaten <patrick.v.platen@gmail.com>
 2021-11-16 08:58:42 -05:00
Lysandre Debut
1cc453d33c Allow per-version configurations (#14344) 
* Allow per-version configurations

* Update tests/test_configuration_common.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update tests/test_configuration_common.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2021-11-15 16:38:02 -05:00
Patrick von Platen
76d0d41e51 [Wav2Vec2] Make sure that gradient checkpointing is only run if needed (#14407) 
* [Wav2Vec2] Make sure that gradient checkpointing is only run if needed

* make fix-copies
 2021-11-15 21:03:10 +01:00
Eldar Kurtic
9fd937ead1 Replace BertLayerNorm with LayerNorm (#14385) 
Running Movement pruning experiments with the newest HuggingFace would crash due to non-existing BertLayerNorm.
 2021-11-15 13:25:10 -05:00
Yih-Dar
a67d47b40c Fix weight loading issue (#14016) 
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2021-11-15 17:48:40 +01:00
NielsRogge
74e6111ba7 Fix test and docs (#14399) 2021-11-15 17:35:33 +01:00
Patrick von Platen
4ce74edf51 [Speech2Text2] Enable tokenizers (#14390) 
* [Speech2Text2] Enable tokenizers

* minor fix

* Apply suggestions from code review

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2021-11-15 16:34:11 +01:00
Matt
267867e851 Quick fix to TF summarization example (#14401) 2021-11-15 13:45:51 +00:00
Stas Bekman
29dfb2dbb1 [doc] performance and parallelism updates (#14391) 
* [doc] performance and parallelism doc update

* improve

* improve
 2021-11-14 17:19:15 -08:00
nbertagnolli
790cdc2e55 Raise exceptions instead of using asserts in modeling_openai #12789 (#14386) 
* Raise exceptions instead of using asserts for control flow in modeling_openai #12789

* reformatted file
 2021-11-13 21:34:34 -05:00
Suraj Patil
2e60276b38 [M2M100Tokenizer] fix _build_translation_inputs (#14382) 
* add return_tensors paramter

* fix test

* Apply suggestions from code review

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* style

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2021-11-13 20:57:12 +05:30
Suraj Patil
3165930402 support wmt21 tokenizer in m2m100 tokenizer (#14376) 2021-11-13 14:21:58 +05:30
Li-Huai (Allan) Lin
280a811ecb Use AlbertConverter for FNet instead of using FNet's own converter (#14365) 
* Add normalizer to FNetConverter

* Style

* Directly use AlbertConverter
 2021-11-12 19:46:40 +01:00
Patrick von Platen
55f49c5f4b [Wav2Vec2 Example] Improve fine-tuning script (#14373) 
* improve some stuff

* finish

* correct last
 2021-11-12 16:35:57 +01:00
Suraj Patil
21546e59a6 fix docs (#14377) 2021-11-12 15:56:41 +05:30
Nicolas Patry
ed5d15518b Adding support for raw python generator in addition to Dataset for pipelines (#14352) 
* Adding support for raw python `generator` in addition to `Dataset`

The main goal is to ease the create of streaming data to the pipe.

`Dataset` is more involved and pytorch specific.

This PR, provides a way to use a python iterator too.
This enabled #14250 but can be proposed as a standalone PR.

```python
from transformers import pipeline

def read_data(filename):
    with open(filename, 'r') as f:
        for line in f:
            yield f

pipe = pipeline("text-classification")
for classified in pipe(read_data("large_file.txt")):
    print("Success ! ", classified)
```

The main caveat of this, is the interaction with `DataLoader` with
`num_workers>1`. When you have multiple workers, each receive a copy
of the generator (like `IterableDataset`). That means the naive Iterator
will fail since all workers iterate on all items of the generator.

There are ways to do clever "skipping", but it could be bad still
because all workers still do have to pass through all items of the
generator (they just ignore items they don't handle), depending on
the case it might be bad.

Using `num_workers=1` is the simplest fix and if the cost of loading
your data is small enough should be good enough. In the above example
trying to do smart tricks to skip some lines is unlikely to be a net
positive for instance.

If there are better ways to do "jumps" on some data, then using
`Dataset` is more advised (since then differents workers can just jump
themselves).

* Adding iterator support for `tf` too.
 2021-11-12 09:20:40 +01:00