Arthur
c236a62172
[CLAP] Add CLAP to the library (#21370)
* add model like clip
* update
* text model ok
* clap text works
* some refactor
- `CLAPVision` to `CLAPAudio`
- refactor kwargs of audio modules
* more refactor
* more refactor
* more refactor
* correct fusion
* more refactor
* new modules
* add basic processor
* fixup
* remove whisper copioed from
* audio logits match
* add doc
* correct filters mel and add maxlength
* style
* few fixes
* forward passes
* fixup
* fixup
* some clean up
* remove mels form the dictionnary
* pad after the repeat
* update padding when dsmaller
* fix padding
* style
* use swin patch merging
* use copied from swin
* processor with any tokenizer
* more copied from
* some clean up
* more refactor
* fix mel when rand_trunc
* style
* remove unused imports
* update processing
* remove image processing tests
* add testing fiel
* fixmodeling issues
* replace with `is_longer`
* clap in serialization
* more refactor
* `make fixup`
* make fixup
* fix feature extractor
* update test feature extractor
* `make fixup`
* clean up config
* more clean up
* more cleanup
* update tests
* refactor tests and inits
* removeCLAP vision config
* remove CLAP from image procssing auto and dummy vision objects
* update inits
* style
* re order classes in modeling clap
* Use roberta tokenizer as the other weights are not open sourced
* small cleaup
* remove tokenization CLAP
* processor tokenizr is roberta
* update feature extraction doc
* remove vclap from model zero shot
* update f_min and f_max to frequency_xx
* some changes
- fix modeling keys
- add `is_longer` in the forward pass
- make fixup
* make fixup
* consistent behavior ebtween rand_crop and fusion
* add numpy resize and bilinear and documentation
* move resizing to image utils
* clean feature extraction
* import resize from correct file
* resize in image transforms
* update
* style
* style
* nit
* remove unused arguments form the feature extractor
* style
* few fixes + make fixup
* oops
* fix more tests
* add zero shot audio classification pipeline
* update zeroshot classification pipeline
* fixup
* fix copies
* all CI tests pass
* make fixup + fix docs
* fix docs
* fix docs
* update tests pip;eline
* update zero shot pipeline
* update feature extraction clap
* update tokenization auto
* use nested simplify
* update pipeline tests
* Apply suggestions from code review
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
* split in two lines
* fixes
* refactor
* clean up
* add integration tests
* update config docstring
* style
* update processor
* fix processor test
* fix feat extractor tests
* update docs
* Apply suggestions from code review
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
* fix readmes
* fix tips
* Update src/transformers/models/auto/configuration_auto.py
* update doc and remove todo -> properly explained
* fix idx and typo
* typoe
* cleanup config
* cleanup tests, styles and doc
* ignore docstyle on image transform
* add conversion script
* remove the `clap` indx in favor of `CLAP`
* update __init
* nits
* Update src/transformers/pipelines/__init__.py
* fix bug
* clarifiy config
* fix copy
* fix init
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* fix model output
* fix comment
* make fixup
* make fixup
* rename to `Clap`
* replace to `Clap`
* replace to `Clap`
* repo consistency
* again repo-consistency
* make fixup
* Apply suggestions from code review
Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
* add config
* changes
* update conversion
* Apply suggestions from code review
Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
* remove unused function
* update based on code reviews
* style
* more comments
* cleanup
* clean up
* style
* apply suggestions
* Empty commit
* pipeline will be added in a different PR
* update calls to audio utils functions
* update pipeline init
* style
* style
* styling again
* use pad
* fix repo-consistency
* update utils and add doc for audio utils
* clean up resize by using torch. update inits accordingly
* style
* CLap's tokenizer is RobertA
* add audio utils to internal toctreee
* update totctree
* style
* update documentation and normalize naming accross audio utils and feature extraction clap
* style
* clean up
* update doc and typos
* fix doctest
* update modelin code, got rid of a lot of reshaping
* style on added doc audio utils
* update modeling clap
* style
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* docstringvariables with CLAP
* rename key
* update modeling CLAP
* update audio utils docstring
* update processing clap
* fix readmes
* fix toctree
* udpate configuration clap
* fix init
* make fixup
* fix
* fix
* update naming
* update
* update checkpoint path
* Apply suggestions from code review
* Major refactoring
* Update src/transformers/models/clap/configuration_clap.py
* merge
---------
Co-authored-by: younesbelkada <younesbelkada@gmail.com>
Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
2023-02-16 20:59:27 +01:00
..
2023-02-15 10:35:14 -08:00
2023-01-04 09:18:57 +01:00
2023-01-17 17:42:33 +01:00
2022-12-16 14:22:46 +01:00
2023-02-15 10:35:14 -08:00
2022-04-04 10:25:46 -04:00
2022-04-04 10:25:46 -04:00
2023-01-17 17:18:56 +01:00
2022-04-29 17:42:15 -04:00
2022-04-04 10:25:46 -04:00
2023-02-15 10:35:14 -08:00
2022-04-04 10:25:46 -04:00
2022-11-04 11:32:44 -04:00
2022-11-04 11:32:44 -04:00
2022-12-05 10:12:03 -05:00
2023-01-17 17:18:56 +01:00
2022-11-04 11:32:44 -04:00
2022-11-04 11:32:44 -04:00
2023-02-09 16:52:11 +01:00
2023-01-18 11:24:37 +01:00
2022-11-07 09:19:04 -05:00
2022-04-04 10:25:46 -04:00
2023-01-25 14:04:32 -05:00
2022-04-13 11:36:54 +02:00
2022-04-04 10:25:46 -04:00
2022-04-04 10:25:46 -04:00
2022-11-30 19:22:23 +01:00
2023-02-16 20:59:27 +01:00
2023-01-17 17:18:56 +01:00
2022-11-09 18:31:22 +01:00
2022-06-24 17:10:38 +02:00
2022-11-30 10:24:03 +00:00
2023-02-15 10:35:14 -08:00
2023-01-17 17:18:56 +01:00
2022-05-02 12:47:39 -04:00
2023-02-15 10:35:14 -08:00
2023-01-17 17:18:56 +01:00
2023-01-17 17:18:56 +01:00
2022-05-10 16:21:44 -04:00
2022-11-15 13:26:07 -05:00
2022-04-04 10:25:46 -04:00
2023-01-17 17:18:56 +01:00
2023-01-17 17:18:56 +01:00
2023-01-31 15:13:12 +01:00
2023-01-17 17:18:56 +01:00
2022-04-04 10:25:46 -04:00
2023-01-17 17:18:56 +01:00
2023-02-15 10:35:14 -08:00
2023-01-17 17:18:56 +01:00
2022-11-29 10:38:01 +00:00
2023-02-15 10:35:14 -08:00
2023-01-17 17:18:56 +01:00
2023-01-20 11:35:42 +03:00
2023-02-15 10:35:14 -08:00
2022-07-27 10:08:59 +02:00
2023-02-15 09:24:56 -05:00
2022-09-09 07:36:46 -04:00
2022-10-31 21:32:58 -04:00
2022-12-27 02:26:14 -05:00
2023-02-15 10:35:14 -08:00
2022-11-08 19:54:41 +00:00
2022-04-04 10:25:46 -04:00
2022-04-04 10:25:46 -04:00
2023-02-15 10:35:14 -08:00
2023-01-17 17:42:33 +01:00
2023-01-17 17:18:56 +01:00
2023-02-15 10:35:14 -08:00
2022-04-04 10:25:46 -04:00
2022-09-14 10:17:40 -04:00
2022-05-24 09:31:10 -04:00
2022-12-12 13:12:13 -05:00
2023-01-30 16:48:04 -05:00
2023-01-19 13:05:59 -05:00
2023-01-17 17:18:56 +01:00
2022-04-04 10:25:46 -04:00
2022-04-04 10:25:46 -04:00
2022-04-04 10:25:46 -04:00
2023-01-17 17:18:56 +01:00
2022-11-10 21:05:27 +01:00
2023-02-03 08:53:16 -05:00
2022-11-08 19:54:41 +00:00
2022-11-08 19:54:41 +00:00
2022-04-04 10:25:46 -04:00
2022-11-04 11:32:44 -04:00
2023-01-17 17:18:56 +01:00
2023-01-17 17:18:56 +01:00
2023-02-15 10:35:14 -08:00
2022-08-16 10:20:46 -05:00
2022-08-01 11:09:47 -04:00
2022-04-04 10:25:46 -04:00
2022-09-29 13:27:58 +02:00
2023-02-15 10:35:14 -08:00
2022-10-24 17:54:23 +02:00
2023-01-25 12:34:43 +01:00
2023-01-16 20:37:07 +03:00
2023-02-15 10:35:14 -08:00
2022-12-27 18:37:25 +01:00
2022-04-04 10:25:46 -04:00
2022-04-04 10:25:46 -04:00
2022-04-04 10:25:46 -04:00
2022-04-04 10:25:46 -04:00
2023-01-17 17:18:56 +01:00
2023-01-17 17:18:56 +01:00
2023-01-17 17:18:56 +01:00
2022-04-04 10:25:46 -04:00
2023-02-15 10:35:14 -08:00
2022-06-29 09:30:55 -04:00
2023-01-17 17:18:56 +01:00
2022-06-23 12:36:22 -04:00
2022-07-29 08:09:09 -04:00
2022-04-04 10:25:46 -04:00
2023-01-25 12:34:43 +01:00
2023-02-15 10:35:14 -08:00
2023-02-16 12:44:28 -05:00
2023-01-03 19:25:09 +03:00
2022-09-02 19:54:02 +02:00
2023-02-15 10:35:14 -08:00
2022-11-08 19:54:41 +00:00
2022-04-04 10:25:46 -04:00
2022-07-29 08:09:09 -04:00
2023-01-17 17:18:56 +01:00
2023-02-15 10:35:14 -08:00
2022-04-04 10:25:46 -04:00
2023-02-15 10:35:14 -08:00
2022-04-04 10:25:46 -04:00
2023-02-15 10:35:14 -08:00
2023-01-17 17:18:56 +01:00
2022-04-04 10:25:46 -04:00
2023-01-17 17:18:56 +01:00
2022-04-04 10:25:46 -04:00
2022-12-19 09:30:17 +01:00
2023-02-15 10:35:14 -08:00
2022-11-08 10:03:43 -05:00
2022-04-04 10:25:46 -04:00
2023-01-17 17:18:56 +01:00
2022-04-04 10:25:46 -04:00
2022-04-04 10:25:46 -04:00
2022-11-09 15:34:08 +00:00
2022-11-03 14:18:45 +01:00
2022-07-29 08:09:09 -04:00
2023-02-03 12:43:46 -05:00
2022-11-07 09:19:04 -05:00
2022-04-04 10:25:46 -04:00
2022-12-16 16:24:01 +01:00
2023-01-17 17:18:56 +01:00
2023-01-17 17:18:56 +01:00
2022-11-15 13:06:45 +01:00
2023-02-15 10:35:14 -08:00
2022-11-07 09:19:04 -05:00
2022-11-30 14:50:55 +00:00
2022-11-14 01:04:26 -05:00
2022-04-08 10:57:51 +02:00
2023-02-13 10:11:16 -05:00
2022-12-02 09:13:25 +01:00
2022-05-17 19:07:43 -04:00
2023-02-15 10:35:14 -08:00
2022-11-30 14:50:55 +00:00
2023-02-15 18:10:30 +00:00
2022-06-21 10:24:50 +02:00
2022-05-17 00:43:16 +02:00
2022-05-17 00:43:16 +02:00
2023-01-31 15:13:12 +01:00
2023-01-17 17:18:56 +01:00
2022-11-30 14:50:55 +00:00
2022-11-08 19:54:41 +00:00
2022-11-30 14:50:55 +00:00
2022-11-07 09:19:04 -05:00
2022-11-30 14:50:55 +00:00
2022-12-07 11:03:39 +01:00
2023-01-17 17:18:56 +01:00
2023-01-17 17:18:56 +01:00
2023-01-17 17:18:56 +01:00
2022-04-04 10:25:46 -04:00
2022-06-15 18:34:15 +02:00
2022-10-28 13:28:18 -07:00
2022-05-17 00:43:16 +02:00
2022-11-09 15:34:08 +00:00
2023-01-17 17:18:56 +01:00
2022-08-24 10:51:05 +01:00
2023-02-15 10:35:14 -08:00
2022-04-04 10:25:46 -04:00
2023-02-15 10:35:14 -08:00
2023-02-07 16:43:19 -05:00
2023-02-15 10:35:14 -08:00
2023-02-15 10:35:14 -08:00
2022-04-04 10:25:46 -04:00
2022-04-04 10:25:46 -04:00
2023-02-16 09:18:25 -05:00
2023-01-17 17:18:56 +01:00
2022-04-04 10:25:46 -04:00