SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity
11,301 Commits 5 Branches 239 Tags
4c7e8d09008ea4e46dd09dccfbd518bb2b792e75
Commit Graph

483 Commits

Author SHA1 Message Date
Younes Belkada
163ac3d3ee Add Switch transformers (#19323) 
* first commit

* add more comments

* add router v1

* clean up

- remove `tf` modeling files

* clean up

- remove `tf` modeling files

* clean up

* v0 routers

* added more router

- Implemented `ExpertsChooseMaskedRouter`

- added tests
- 2 more routers to implement

* last router

* improved docstring

- completed the docstring in `router.py`
- added more args in the config

* v0 sparse mlp

* replace wrong naming

* forward pass run

* update MOE layer

* small router update

* fixup

* consistency

* remove scatter router

* remove abstract layer

* update test and model for integration testing

* v1 conversion

* update

* hardcode hack

* all keys match

* add gin conversion, without additional libraries

* update conversion sctipy

* delete router file

* update tests wrt router deletion

* fix router issues

* update expert code

* update, logits match, code needsREFACTORING

* Refactor code

Co-authored-by: Younes Belkada <younesbelkada@users.noreply.github.com>

* add generate tests

Co-authored-by: younesbelkada <younesbelkada@gmail.com>

* add support for router loss

Co-authored-by: Younes Belkada <younesbelkada@users.noreply.github.com>

* fix forward error

* refactor a bit

* remove `FlaxSwitchTransformers` modules

* more tests pass

* Update code

Co-authored-by: Younes Belkada <younesbelkada@users.noreply.github.com>

* fixup

* fix tests

* fix doc

* fix doc + tokenization

* fix tokenizer test

* fix test

* fix loss output

* update code for backward pass

* add loss support

* update documentation

* fix documentation, clean tokenizer

* more doc fix, cleanup example_switch

* fix failing test

* fix test

* fix test

* fix loss issue

* move layer

* update doc and fix router capacity usage

* fixup

* add sparse mlp index for documentation on hub

* fixup

* test sparse mix architecture

* Apply suggestions from code review

* Update docs/source/en/model_doc/switch_transformers.mdx

* fixup on update

* fix tests

* fix another test

* attempt fix

* Update src/transformers/models/switch_transformers/configuration_switch_transformers.py

Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>

* Update src/transformers/models/switch_transformers/convert_switch_transformers_original_flax_checkpoint_to_pytorch.py

Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>

* try

* all tests pass

* fix jitter noise

* Apply suggestions from code review

* doc tests pass

* Update src/transformers/models/switch_transformers/modeling_switch_transformers.py

Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>

* Update src/transformers/models/switch_transformers/modeling_switch_transformers.py

Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>

* remove assert

* change config order

* fix readme japanese

* Apply suggestions from code review

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* remove parallelizable tests + add one liners

* remove ONNX config

* fix nits

- add `T5Tokenizer` in auto mapping
- remove `Switch Transformers` from ONNX supported models

* remove `_get_router`

* remove asserts

* add check in test for `router_dtype`

* add `SwitchTransformersConfig` in `run_pipeline_test`

* Update tests/pipelines/test_pipelines_summarization.py

* add huge model conversion script

* fix slow tests

- add better casting for `Linear8bitLt`
- remove `torchscript` tests

* add make dir

* style on new script

* fix nits

- doctest
- remove `_keys_to_ignore_on_load_unexpected`

* Update src/transformers/models/switch_transformers/configuration_switch_transformers.py

* add google as authors

* fix year

* remove last `assert` statements

* standardize vertical spaces

* fix failing import

* fix another failing test

* Remove strange àuthorized_keys`

* removing todo and padding that is never used

Co-authored-by: Arthur Zucker <arthur.zucker@gmail.com>
Co-authored-by: ybelkada <younes@huggingface.co>
Co-authored-by: Younes Belkada <younesbelkada@users.noreply.github.com>
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Arthur Zucker <arthur@huggingface.co>
 2022-11-15 13:06:45 +01:00
Matthijs Hollemans
f711d683b5 add MobileNetV2 model (#17845) 
* add model files etc for MobileNetV2

* rename files for MobileNetV1

* initial implementation of MobileNetV1

* fix conversion script

* cleanup

* write docs

* tweaks

* fix conversion script

* extract hidden states

* fix test cases

* make fixup

* fixup it all

* rename V1 to V2

* fix checkpoints

* fixup

* implement first block + weight conversion

* add remaining layers

* add output stride and dilation

* fixup

* add tests

* add deeplabv3+ head

* a bit of fixup

* finish deeplab conversion

* add link to doc

* fix issue with JIT trace

in_height and in_width would be Tensor objects during JIT trace, which caused Core ML conversion to fail on the remainder op. By making them ints, the result of the padding calculation becomes a constant value.

* cleanup

* fix order of models

* fix rebase error

* remove main from doc link

* add image processor

* remove old feature extractor

* fix converter + other issues

* fixup

* fix unit test

* add to onnx tests (but these appear broken now)

* add post_process_semantic_segmentation

* use google org

* remove unused imports

* move args

* replace weird assert
 2022-11-14 01:00:10 -05:00
Arthur
61a51f5f23 Add Jukebox model (replaces #16875) (#17826) 2022-11-10 21:05:27 +01:00
NielsRogge
9f0c72f93b Add doc tests (#20158) 
Co-authored-by: Niels Rogge <nielsrogge@Nielss-MBP.localdomain>
 2022-11-10 15:25:30 +01:00
Joao Gante
f270b960d6 Generate: move generation_*.py src files into generation/*.py (#20096) 
* move generation_*.py src files into generation/*.py

* populate generation.__init__ with lazy loading

* move imports and references from generation.xxx.object to generation.object
 2022-11-09 15:34:08 +00:00
Yih-Dar
316bf04d3d Improve tiny model creation script (#20119) 
* Improve tiny model creation script

* sort the list of models to upload

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2022-11-09 11:34:35 +01:00
amyeroberts
4eb918e656 AutoImageProcessor (#20111) 
* AutoImageProcessor skeleton

* Update references

* Add mapping in init

* Add model image processors to __init__ for importing

* Add AutoImageProcessor tests

* Fix up

* Image Processor documentation

* Remove pdb

* Update docs/source/en/model_doc/mobilevit.mdx

* Update docs

* Don't add whitespace on json files

* Remove fixtures

* Move checking model config down

* Fix up

* Add check for image processor

* Remove FeatureExtractorMixin in docstrings

* Rename model_tmpfile to config_tmpfile

* Don't make None if not in image processor map
 2022-11-08 19:54:41 +00:00
Weiwe Shi
efa889d2e4 Add RocBert (#20013) 
* add roc_bert

* update roc_bert readme

* code style

* change name and delete unuse file

* udpate model file

* delete unuse log file

* delete tokenizer fast

* reformat code and change model file path

* add RocBertForPreTraining

* update docs

* delete wrong notes

* fix copies

* fix make repo-consistency error

* fix files are not present in the table of contents error

* change RocBert -> RoCBert

* add doc, add detail test

Co-authored-by: weiweishi <weiweishi@tencent.com>
 2022-11-08 10:03:43 -05:00
NielsRogge
258963062b Add CLIPSeg (#20066) 
* Add first draft

* Update conversion script

* Improve conversion script

* Improve conversion script some more

* Add conditional embeddings

* Add initial decoder

* Fix activation function of decoder

* Make decoder outputs match original implementation

* Make decoder outputs match original implementation

* Add more copied from statements

* Improve model outputs

* Fix auto tokenizer file

* Fix more tests

* Add test

* Improve README and docs, improve conditional embeddings

* Fix more tests

* Remove print statements

* Remove initial embeddings

* Improve conversion script

* Add interpolation of position embeddings

* Finish addition of interpolation of position embeddings

* Add support for refined checkpoint

* Fix refined checkpoint

* Remove unused parameter

* Improve conversion script

* Add support for training

* Fix conversion script

* Add CLIPSegFeatureExtractor

* Fix processor

* Fix CLIPSegProcessor

* Fix conversion script

* Fix most tests

* Fix equivalence test

* Fix README

* Add model to doc tests

* Use better variable name

* Convert other checkpoint as well

* Update config, add link to paper

* Add docs

* Update organization

* Replace base_model_prefix with clip

* Fix base_model_prefix

* Fix checkpoint of config

* Fix config checkpoint

* Remove file

* Use logits for output

* Fix tests

Co-authored-by: Niels Rogge <nielsrogge@Nielss-MacBook-Pro.local>
 2022-11-08 10:55:47 +01:00
Saad Mahmud
9617b1304e [Doctest] Add configuration_dpr.py (#20080) 
* Add example docstring for DPRConfig

* Add DPRConfig to documentation_tests
 2022-11-07 14:49:59 +01:00
Saad Mahmud
790ff2544a [Doctest] Add configuration_camembert.py (#20039) 
* Add example docstring for CamembertConfig

* Add configuration_camembert to documentation_tests
 2022-11-03 14:50:42 +01:00
Saad Mahmud
7487743793 [Doctest] Add configuration_deberta_v2.py (#19995) 
* Add example docstring for DebertaV2Config

* Add DebertaV2Config to documentation_tests

* Fix mistake with directory name
 2022-11-02 16:22:11 +01:00
Ikko Ashimine
38e5b71abb Add Japanese translated README (#19945) 
* Add japanese translated README.md

* Add README_ja.md link

* Add japanese transkate to check_copies.py

* Add guide to Japanese README.md

* Update README_ja.md

Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>

* Update utils/check_copies.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2022-11-01 09:18:08 -04:00
Matt
7f9b7b3f0e Add ESMFold (#19977) 
* initial commit

* First draft that gets outputs without crashing!

* Add all the ported openfold dependencies

* testing

* Restructure config files for ESMFold

* Debugging to find output discrepancies

* Mainly style

* Make model runnable without extra deps

* Remove utils and merge them to the modeling file

* Use correct gelu and remove some debug prints

* More cleanup

* Update esm docs

* Update conversion script to support ESMFold properly

* Port some top-level changes from ESMFold repo

* Expand EsmFold docstrings

* Make attention_mask optional (default to all 1s)

* Add inference test for ESMFold

* Use config and not n kwargs

* Add modeling output class

* Remove einops

* Remove chunking in ESM FFN

* Update tests for ESMFold

* Quality

* REpo consistency

* Remove tree dependency from ESMFold

* make fixup

* Add an error in case my structure map function breaks later

* Remove needless code

* Stop auto-casting the LM to float16 so CPU tests pass

* Stop auto-casting the LM to float16 so CPU tests pass

* Final test updates

* Split test file

* Copyright and quality

* Unpin PyTorch to see built doc

* Fix config file to_dict() method

* Add some docstrings to the output

* Skip TF checkpoint tests for ESM until we reupload those

* make fixup

* More docstrings

* Unpin to get even with main

* Flag example to write

Co-authored-by: Sylvain Gugger <Sylvain.gugger@gmail.com>
 2022-10-31 21:32:58 -04:00
Saad Mahmud
6176e13612 [Doctest] Add configuration_deberta.py (#19968) 
* Add Example docstring to DebertaConfig

* Add configuration_deberta to documentation_tests

* Add microsoft/deberta-base to example docstring

* Fix example docstring mistake
 2022-10-31 17:22:01 +01:00
Yih-Dar
dcca71be61 Create dummy models (#19901) 
* create dummy models

* quality

* update

* update

* Make Wav2Vec2Conformer work

* style

* deal with models with text_config and vision_config

* apply suggestions

* Composite models

* style

* style

* fix shape issue

* fix shape issue

* For VisionTextDualEncoderModel

* show_progress=False when converting tokenizers

* Fix for OwlViT

* Fix for VisualBert

* Update

* final

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2022-10-28 13:05:41 +02:00
Yih-Dar
803475fb69 Add checkpoint links in a few config classes (#19910) 
* For CLIP

* Others

* update

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2022-10-27 09:26:10 +02:00
Revanth09
1f1cc09df6 [DOCTEST] Config doctest for MCTCT, MBart and LayoutLM (#19889) 
* Update documentation_tests.txt

* Update configuration_mbart.py

* Update configuration_mctct.py

* Update configuration_layoutlm.py

* Update configuration_layoutlmv2.py

* Update configuration_layoutlmv3.py

* Update documentation_tests.txt
 2022-10-26 12:05:44 +02:00
Rak Alexey
1f7e40d04f Improve check copies (#19829) 
* print first diff line intead of first code part line

* fix style
 2022-10-24 11:24:18 -04:00
Ayaka
536f338441 [Doctest] Add configuration_nezha.py (#19810) 
* [Doctest] Add `configuration_nezha.py`

* Revert line order
 2022-10-24 13:50:43 +02:00
Ayaka
f58b211ed3 [Doctest] Add configuration_electra.py (#19807) 2022-10-24 12:34:43 +02:00
Ayaka
c949188b9d [Doctest] Add configuration_poolformer.py (#19808) 2022-10-24 12:33:46 +02:00
Ayaka
82df83a96b [Doctest] Add configuration_plbart.py (#19809) 
Additionally, I updated the doctest format to be consistent with BERT.
 2022-10-24 12:32:55 +02:00
sha016
22502ebb85 [Doctest] MaskFormerConfig doctest (#19817) 2022-10-24 11:08:32 +02:00
Rohith ND
eb98da9880 [Doctest] OpenAIGPTConfig and OPTConfig (#19763) 2022-10-20 10:22:00 +02:00
Partho
506355ca75 [Doctest] SpeechToTextTransformer2 Config for doctest (#19756) 2022-10-20 10:19:06 +02:00
Partho
123f65eea6 [Doctest] SqueezeBERT Config for doctest (#19758) 2022-10-20 10:16:39 +02:00
Partho
cc03063366 [Doctest] SpeechToTextTransformer Config for doctest (#19757) 2022-10-20 10:15:07 +02:00
Xabier Lahuerta Vazquez
0a03741590 [Doctest] Add configuration_detr.py (#19752) 2022-10-19 18:13:34 +02:00
Xabier Lahuerta Vazquez
65d36ee861 [Doctest] Add configuration_decision_transformer.py (#19751) 2022-10-19 18:12:34 +02:00
JuanCarlosPi
c206fc8779 [Doctest] Add configuration_wavlm.py (#19749) 
* Change the import order of the model and configuration classes

* Add (with random weights) in the comment before model initialization

* Add configuration_wavlm to doctest
 2022-10-19 16:10:13 +02:00
Rohith ND
7df0751cc6 [Doctest] GPTNeoConfig , GPTNeoXConfig , GPTNeoXJapaneseConfig (#19741) 2022-10-19 14:22:41 +02:00
Partho
fc5fdc109d [Doctest] Add configuration_clip.py (#19647) 
* CLIP Config for doctest

* add doc example to CLIPConfig

* add from_text_vision_configs example

* added comment explaining objective
 2022-10-19 09:51:26 +02:00
AymenBer99
c9a0da1e12 [Doctest] XLM Config for doctest (#19685) 2022-10-19 07:10:30 +02:00
JuanCarlosPi
eccbdbcd4d [Doctest] Add wav2vec2_conformer for doctest (#19734) 2022-10-19 06:47:41 +02:00
Sylvain Gugger
ebee0a2794 Remove debug statement 2022-10-18 13:58:09 -04:00
Rohith ND
fa8ed9ca76 [Doctest] Add doctest for FlavaConfig and FNetConfig (#19724) 2022-10-18 19:56:49 +02:00
Sylvain Gugger
a929f81e92 Repo utils test (#19696) 
* Create repo utils test job

* Last occurence

* Add tests for tests_fetcher

* Better filtering

* Let's learn more

* Should fix

* Should fix

* Remove debug

* Style

* WiP

WiP

WiP

WiP

WiP

WiP

WiP

WiP

WiP

* Quality

* address review comments

* Fix link
 2022-10-18 13:47:36 -04:00
Mukesh K
f3ed26a3fb [Doctest] Fixing doctest configuration_pegasus_x.py (#19725) 
* Fixed pegasus_x config doctest

* Test commit

Co-authored-by: mukesh663 <mukesh13034@gmail.com>
 2022-10-18 17:19:31 +02:00
Somasree Majumder
5864051109 [Doctest] Adding config files for convnext (#19717) 
* Adding config files for configuration_clip.py

* Adding config files for convnext

* Undoing

* making the required changes

* Update documentation_tests.txt
 2022-10-18 17:10:09 +02:00
Mukesh K
fac1f4b188 ]Fixed pegasus config doctest (#19722) 
Co-authored-by: mukesh663 <mukesh13034@gmail.com>
 2022-10-18 15:38:57 +02:00
NielsRogge
dd523da577 Add table transformer [v2] (#19614) 
* First draft

* Add conversion script

* Make conversion work

* Upload checkpoints

* Add final fixes

* Revert changes of conditional and deformable detr

* Fix toctree, add and remove copied from

* Use model type

* Improve docs

* Improve code example

* Update copies

* Add copied formt

* Don't update conditional detr

* Don't update deformable detr
 2022-10-18 15:20:09 +02:00
JuanCarlosPi
fd99ce3329 [Doctest] Add configuration_wav2vec2.py to documentation_tests.py (#19698) 2022-10-18 14:57:34 +02:00
AymenBer99
8fcbbd3d53 [Doctest] CVT config for doctest (#19695) 2022-10-18 14:55:56 +02:00
Partho
2add2007c1 [Doctest] Add configuration_data2vec_vision.py (#19637) 
* Data2Vec Vision Config for doctest

* made suggested changes

Co-authored-by: Yih-Dar <2521628+ydshieh@users.noreply.github.com>
 2022-10-17 20:56:42 +02:00
Akshita Prasanth
684165b882 [Doctest] Add configuration_realm.py (#19646) 
* Update configuration_realm.py

* realm config for doctest

* Update configuration_realm.py doc

* Update documentation_tests

* clean up

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2022-10-17 20:53:24 +02:00
AymenBer99
5ac2f82267 [Doctest] Add configuration_convbert.py (#19643) 
* ConvBERT config for doctest

* Add empty lines
 2022-10-17 20:29:18 +02:00
Partho
4d77f18cba [Doctest] Data2VecAudio Config for doctest (#19635) 2022-10-17 18:39:15 +02:00
Thomas
0027edf905 [Doctest] Add configuration_transfo_xl.py (#19651) 
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2022-10-17 16:47:54 +02:00
Thomas
9bb26f2505 [Doctest] Add configuration_trocr.py (#19658) 
* trocr Config for doctest

* ran make style
 2022-10-17 10:53:36 +02:00