Matthijs Hollemans
0c3fdccf2f
[WIP] add EnCodec model (#23655)
* boilerplate stuff
* messing around with the feature extractor
* fix feature extractor
* unit tests for feature extractor
* rename speech to audio
* quick-and-dirty import of Meta's code
* import weights (sort of)
* cleaning up
* more cleaning up
* move encoder/decoder args into config
* cleanup model
* rename EnCodec -> Encodec
* RVQ parameters in config
* add slow test
* add lstm init and test_init
* Add save & load
* finish EncodecModel
* remove decoder_input_values as they are ont used anywhere (not removed from doc yet)
* fix test feature extraction model name
* Add better slow test
* Fix tests
* some fixup and cleaning
* Improve further
* cleaning up quantizer
* fix up conversion script
* test don't pass, _encode_fram does not work
* update tests with output per encode and decode
* more cleanup
* rename _codebook
* remove old config cruft
* ratios & hop_length
* use ModuleList instead of Sequential
* clean up resnet block
* update types
* update tests
* fixup
* quick cleanup
* fix padding
* more styl,ing
* add patrick feedback
* fix copies
* fixup
* fix lstm
* fix shape issues
* fixup
* rename conv layers
* fixup
* fix decoding
* small conv refactoring
* remove norm_params
* simplify conv layers
* rename conv layers
* stuff
* Clean up
* Add padding logic
use padding mask
small conv refactoring
remove norm_params
simplify conv layers
rename conv layers
stuff
add batched test
update
Clean up
merge and update for padding
fix padding
fixup
* clean up more
* clean up more
* More clean ups
* cleanup convolutions
* typo
* fix typos
* fixup
* build PR doc?
* start refactoring docstring
* fix don't pad when no strid and chunk
* update docstring
* update docstring
* nits
* update going to lunch
* update config and model
* fix broken testse (becaue of the config changes)
* fix scale computation
* fixu[
* only return dict if speciefied or if config returns it
* remove todos
* update defaults in config
* update conversion script
* fix doctest
* more docstring + fixup
* nits on batched_tests
* more nits
* Apply suggestions from code review
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
* update basxed on review
* fix update
* updaet tests
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* fixup
* add overlap and chunl_length_s
* cleanup feature extraction
* teste edge cases truncation and padding
* correct processor values
* update config encodec, nits
* fix tests
* fixup
* fix 24Hz test
* elle tests are green
* fix fixup
* Apply suggestions from code review
* revert readme changes
* fixup
* add example
* use facebook checkpoints
* fix typo
* no pipeline tests
* use slef.pad everywhere we can
* Apply suggestions from code review
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* update based on review
* update
* update mdx
* fix bug and tests
* fixup
* fix doctest
* remove comment
* more nits
* add more coverage for `test_truncation_and_padding`
* fixup
* add last test
* fix text
* nits
* Update tests/models/encodec/test_modeling_encodec.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* take care of the last comments
* typo
* fix test
* nits
* fixup
* Update src/transformers/models/encodec/feature_extraction_encodec.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: arthur.zucker@gmail.com <arthur.zucker@gmail.com>
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
2023-06-14 18:57:23 +02:00
..
2023-03-17 14:30:17 +00:00
2023-03-16 13:41:48 +03:00
2023-01-04 09:18:57 +01:00
2023-03-17 14:30:17 +00:00
2023-04-21 10:04:45 -04:00
2023-05-30 10:23:32 +02:00
2023-03-17 14:30:17 +00:00
2022-04-04 10:25:46 -04:00
2022-04-04 10:25:46 -04:00
2023-03-17 14:30:17 +00:00
2022-04-29 17:42:15 -04:00
2022-04-04 10:25:46 -04:00
2023-03-17 14:30:17 +00:00
2022-04-04 10:25:46 -04:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-05-01 09:17:27 -04:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-02-28 15:42:55 +01:00
2023-04-04 16:05:22 +01:00
2023-03-17 14:30:17 +00:00
2022-04-04 10:25:46 -04:00
2023-03-08 09:00:54 -05:00
2022-04-13 11:36:54 +02:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2022-11-30 19:22:23 +01:00
2023-04-24 14:00:29 +02:00
2023-01-17 17:18:56 +01:00
2022-11-09 18:31:22 +01:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-03-14 12:08:14 +03:00
2022-05-02 12:47:39 -04:00
2023-04-12 07:33:20 -04:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2022-04-04 10:25:46 -04:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-04-05 17:43:48 +02:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2022-04-04 10:25:46 -04:00
2023-03-17 14:30:17 +00:00
2023-05-03 18:23:09 +02:00
2023-01-17 17:18:56 +01:00
2022-11-29 10:38:01 +00:00
2023-02-15 10:35:14 -08:00
2023-03-17 14:30:17 +00:00
2023-05-31 10:43:12 +01:00
2023-02-20 16:37:11 +03:00
2023-03-17 14:30:17 +00:00
2023-06-14 18:57:23 +02:00
2022-07-27 10:08:59 +02:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2022-12-27 02:26:14 -05:00
2023-03-10 07:44:45 -05:00
2023-03-17 14:30:17 +00:00
2022-11-08 19:54:41 +00:00
2023-03-17 14:30:17 +00:00
2023-04-23 20:03:05 +03:00
2022-04-04 10:25:46 -04:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-03-17 10:27:12 -04:00
2023-05-03 18:23:09 +02:00
2023-04-10 10:57:21 +02:00
2023-05-03 15:59:19 -04:00
2023-03-17 14:30:17 +00:00
2023-05-04 10:15:15 -04:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-02-20 11:25:27 +01:00
2023-05-15 12:04:16 +01:00
2023-01-17 17:18:56 +01:00
2022-04-04 10:25:46 -04:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-06-02 12:32:34 -04:00
2023-02-27 08:36:36 +01:00
2023-03-17 14:30:17 +00:00
2023-05-03 09:53:00 -04:00
2023-03-17 14:30:17 +00:00
2022-04-04 10:25:46 -04:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-06-09 16:30:20 +02:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-01-25 12:34:43 +01:00
2023-01-16 20:37:07 +03:00
2023-04-05 17:43:48 +02:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-04-25 17:58:45 -04:00
2022-04-04 10:25:46 -04:00
2023-03-17 14:30:17 +00:00
2023-03-17 09:40:06 +00:00
2022-04-04 10:25:46 -04:00
2023-06-02 10:30:24 +01:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-06-02 10:37:02 +01:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-04-21 17:09:40 +01:00
2023-04-04 14:53:06 +02:00
2023-03-17 14:30:17 +00:00
2023-01-25 12:34:43 +01:00
2023-06-09 12:32:16 +01:00
2023-03-17 14:30:17 +00:00
2023-05-26 14:30:32 +02:00
2023-01-03 19:25:09 +03:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2022-04-04 10:25:46 -04:00
2023-05-16 14:49:24 +02:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-02-15 10:35:14 -08:00
2022-04-04 10:25:46 -04:00
2023-03-17 14:30:17 +00:00
2023-04-04 12:41:12 -04:00
2023-03-17 14:30:17 +00:00
2023-03-24 19:45:57 +00:00
2022-04-04 10:25:46 -04:00
2023-03-17 14:30:17 +00:00
2023-05-03 18:23:09 +02:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-05-09 13:04:10 -04:00
2023-05-19 14:14:13 +01:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2022-11-03 14:18:45 +01:00
2022-07-29 08:09:09 -04:00
2023-02-03 12:43:46 -05:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-05-12 11:52:31 +01:00
2022-12-16 16:24:01 +01:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-05-03 18:23:09 +02:00
2022-11-07 09:19:04 -05:00
2023-02-21 10:35:11 -05:00
2023-03-17 14:30:17 +00:00
2022-04-08 10:57:51 +02:00
2023-06-02 12:32:34 -04:00
2023-03-17 14:30:17 +00:00
2022-05-17 19:07:43 -04:00
2023-03-17 14:30:17 +00:00
2023-05-05 11:29:20 -04:00
2023-02-15 18:10:30 +00:00
2022-06-21 10:24:50 +02:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2022-11-08 19:54:41 +00:00
2022-11-30 14:50:55 +00:00
2023-03-01 18:00:48 +00:00
2022-11-30 14:50:55 +00:00
2023-03-17 14:30:17 +00:00
2023-01-17 17:18:56 +01:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2022-04-04 10:25:46 -04:00
2023-03-17 14:30:17 +00:00
2023-06-02 11:35:52 +01:00
2023-03-17 14:30:17 +00:00
2023-05-05 13:23:46 -04:00
2023-01-17 17:18:56 +01:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-02-07 16:43:19 -05:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2022-04-04 10:25:46 -04:00
2022-04-04 10:25:46 -04:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00