Matthijs Hollemans
0c3fdccf2f
[WIP] add EnCodec model (#23655)
* boilerplate stuff
* messing around with the feature extractor
* fix feature extractor
* unit tests for feature extractor
* rename speech to audio
* quick-and-dirty import of Meta's code
* import weights (sort of)
* cleaning up
* more cleaning up
* move encoder/decoder args into config
* cleanup model
* rename EnCodec -> Encodec
* RVQ parameters in config
* add slow test
* add lstm init and test_init
* Add save & load
* finish EncodecModel
* remove decoder_input_values as they are ont used anywhere (not removed from doc yet)
* fix test feature extraction model name
* Add better slow test
* Fix tests
* some fixup and cleaning
* Improve further
* cleaning up quantizer
* fix up conversion script
* test don't pass, _encode_fram does not work
* update tests with output per encode and decode
* more cleanup
* rename _codebook
* remove old config cruft
* ratios & hop_length
* use ModuleList instead of Sequential
* clean up resnet block
* update types
* update tests
* fixup
* quick cleanup
* fix padding
* more styl,ing
* add patrick feedback
* fix copies
* fixup
* fix lstm
* fix shape issues
* fixup
* rename conv layers
* fixup
* fix decoding
* small conv refactoring
* remove norm_params
* simplify conv layers
* rename conv layers
* stuff
* Clean up
* Add padding logic
use padding mask
small conv refactoring
remove norm_params
simplify conv layers
rename conv layers
stuff
add batched test
update
Clean up
merge and update for padding
fix padding
fixup
* clean up more
* clean up more
* More clean ups
* cleanup convolutions
* typo
* fix typos
* fixup
* build PR doc?
* start refactoring docstring
* fix don't pad when no strid and chunk
* update docstring
* update docstring
* nits
* update going to lunch
* update config and model
* fix broken testse (becaue of the config changes)
* fix scale computation
* fixu[
* only return dict if speciefied or if config returns it
* remove todos
* update defaults in config
* update conversion script
* fix doctest
* more docstring + fixup
* nits on batched_tests
* more nits
* Apply suggestions from code review
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
* update basxed on review
* fix update
* updaet tests
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* fixup
* add overlap and chunl_length_s
* cleanup feature extraction
* teste edge cases truncation and padding
* correct processor values
* update config encodec, nits
* fix tests
* fixup
* fix 24Hz test
* elle tests are green
* fix fixup
* Apply suggestions from code review
* revert readme changes
* fixup
* add example
* use facebook checkpoints
* fix typo
* no pipeline tests
* use slef.pad everywhere we can
* Apply suggestions from code review
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* update based on review
* update
* update mdx
* fix bug and tests
* fixup
* fix doctest
* remove comment
* more nits
* add more coverage for `test_truncation_and_padding`
* fixup
* add last test
* fix text
* nits
* Update tests/models/encodec/test_modeling_encodec.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* take care of the last comments
* typo
* fix test
* nits
* fixup
* Update src/transformers/models/encodec/feature_extraction_encodec.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: arthur.zucker@gmail.com <arthur.zucker@gmail.com>
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
2023-06-14 18:57:23 +02:00
..
2023-06-13 17:51:37 +01:00
2023-05-15 16:46:58 +02:00
2023-03-23 19:14:17 +01:00
2023-05-23 14:37:35 -04:00
2023-06-06 14:31:14 -04:00
2023-05-30 10:23:32 +02:00
2023-06-13 17:51:37 +01:00
2023-02-06 18:10:56 -05:00
2023-02-06 18:10:56 -05:00
2023-02-28 19:40:57 +01:00
2023-06-13 17:51:37 +01:00
2023-02-28 19:40:57 +01:00
2023-02-06 18:10:56 -05:00
2023-02-06 18:10:56 -05:00
2023-05-07 18:55:04 -04:00
2023-05-18 11:04:51 +01:00
2023-05-18 17:29:04 +02:00
2023-04-06 13:50:15 +01:00
2023-06-13 17:51:37 +01:00
2023-06-13 17:51:37 +01:00
2023-05-24 13:52:52 +01:00
2023-03-13 15:03:21 +01:00
2023-05-16 19:28:19 +01:00
2023-05-24 13:52:52 +01:00
2023-05-23 17:32:57 +02:00
2023-02-06 18:10:56 -05:00
2023-05-24 13:52:52 +01:00
2023-02-28 19:40:57 +01:00
2023-02-28 19:40:57 +01:00
2023-05-23 14:37:35 -04:00
2023-05-24 13:52:52 +01:00
2023-02-28 19:40:57 +01:00
2023-04-24 15:27:31 +02:00
2023-03-07 14:23:36 +01:00
2023-05-24 13:52:52 +01:00
2023-05-24 13:52:52 +01:00
2023-05-16 23:35:11 +02:00
2023-03-22 20:02:24 +01:00
2023-05-18 17:29:04 +02:00
2023-06-14 14:39:02 +01:00
2023-05-24 13:52:52 +01:00
2023-05-24 13:52:52 +01:00
2023-05-24 13:52:52 +01:00
2023-05-24 13:52:52 +01:00
2023-02-28 19:40:57 +01:00
2023-02-28 19:40:57 +01:00
2023-05-24 13:52:52 +01:00
2023-04-13 21:32:29 +02:00
2023-03-07 15:19:39 -05:00
2023-04-06 13:50:15 +01:00
2023-05-24 13:52:52 +01:00
2022-05-03 14:42:02 +02:00
2023-02-28 19:40:57 +01:00
2023-05-24 13:52:52 +01:00
2023-02-28 19:40:57 +01:00
2023-05-31 10:43:12 +01:00
2023-04-06 22:52:59 +02:00
2023-05-24 13:52:52 +01:00
2023-06-14 18:57:23 +02:00
2023-06-06 18:30:51 +01:00
2023-02-28 19:40:57 +01:00
2023-03-23 19:14:17 +01:00
2023-05-24 13:52:52 +01:00
2023-05-24 13:52:52 +01:00
2023-02-28 19:40:57 +01:00
2023-04-19 20:27:19 +02:00
2023-05-18 17:29:04 +02:00
2023-04-06 17:56:06 +02:00
2023-05-24 13:52:52 +01:00
2023-02-28 19:40:57 +01:00
2023-02-28 19:40:57 +01:00
2023-06-13 17:51:37 +01:00
2023-06-13 19:19:40 +02:00
2023-05-18 17:29:04 +02:00
2023-06-12 15:37:12 +02:00
2023-02-28 19:40:57 +01:00
2022-12-12 13:12:13 -05:00
2023-06-13 20:33:26 +02:00
2023-04-06 22:52:59 +02:00
2023-02-28 19:40:57 +01:00
2023-05-24 13:52:52 +01:00
2023-02-06 18:10:56 -05:00
2023-05-24 13:52:52 +01:00
2023-02-28 19:40:57 +01:00
2023-05-18 11:04:51 +01:00
2023-05-30 10:23:32 +02:00
2023-03-07 04:20:14 +01:00
2023-05-24 13:52:52 +01:00
2023-04-17 17:45:20 +02:00
2023-05-24 13:52:52 +01:00
2023-02-22 09:14:54 +01:00
2023-06-13 17:51:37 +01:00
2023-02-28 19:40:57 +01:00
2023-03-14 10:03:02 +01:00
2023-05-30 16:50:41 +02:00
2023-05-24 13:52:52 +01:00
2023-04-06 17:56:06 +02:00
2023-03-22 20:02:24 +01:00
2023-06-13 17:51:37 +01:00
2023-04-06 17:56:06 +02:00
2023-06-13 17:51:37 +01:00
2023-04-11 10:00:34 +02:00
2023-03-22 20:45:08 -04:00
2023-04-06 13:50:15 +01:00
2023-06-13 17:51:37 +01:00
2022-07-29 08:09:09 -04:00
2023-05-23 14:37:35 -04:00
2023-04-07 17:13:04 +02:00
2023-02-28 19:40:57 +01:00
2022-05-03 14:42:02 +02:00
2023-04-06 17:56:06 +02:00
2022-05-12 16:25:55 -04:00
2023-06-13 17:51:37 +01:00
2023-02-28 19:40:57 +01:00
2023-02-28 19:40:57 +01:00
2023-05-24 13:52:52 +01:00
2023-06-05 18:12:45 +02:00
2023-05-24 13:52:52 +01:00
2023-05-24 13:52:52 +01:00
2023-05-18 11:04:51 +01:00
2023-04-06 13:50:15 +01:00
2023-02-28 19:40:57 +01:00
2023-04-04 14:53:06 +02:00
2023-04-21 17:09:40 +01:00
2023-02-28 19:40:57 +01:00
2023-03-23 19:14:17 +01:00
2023-04-28 11:01:32 -04:00
2023-06-13 17:51:37 +01:00
2023-06-13 17:51:37 +01:00
2023-02-28 19:40:57 +01:00
2023-06-13 17:51:37 +01:00
2023-04-06 17:56:06 +02:00
2023-02-28 19:40:57 +01:00
2023-02-06 18:10:56 -05:00
2023-06-14 17:05:40 +02:00
2023-05-18 11:04:51 +01:00
2023-02-28 19:40:57 +01:00
2023-05-18 11:04:51 +01:00
2023-02-28 19:40:57 +01:00
2023-06-13 19:04:22 +01:00
2023-05-31 14:59:30 +01:00
2023-05-18 11:04:51 +01:00
2023-05-24 13:52:52 +01:00
2023-05-24 13:52:52 +01:00
2023-05-24 13:52:52 +01:00
2023-02-06 18:10:56 -05:00
2023-05-24 13:52:52 +01:00
2023-05-24 13:52:52 +01:00
2023-03-14 10:03:02 +01:00
2023-05-24 13:52:52 +01:00
2023-06-12 18:14:15 +02:00
2023-06-13 19:04:22 +01:00
2023-05-24 13:52:52 +01:00
2023-02-28 19:40:57 +01:00
2023-02-28 19:40:57 +01:00
2023-02-06 18:10:56 -05:00
2023-05-24 13:52:52 +01:00
2023-02-28 19:40:57 +01:00
2023-05-23 14:37:35 -04:00
2023-04-06 17:45:55 +02:00
2023-02-28 19:40:57 +01:00
2023-05-18 17:29:04 +02:00
2023-05-24 13:52:52 +01:00
2023-03-07 14:23:36 +01:00
2023-03-07 14:23:36 +01:00
2023-04-06 17:56:06 +02:00
2023-06-13 17:51:37 +01:00
2023-02-28 19:40:57 +01:00
2023-06-13 19:19:40 +02:00
2023-02-06 18:10:56 -05:00
2023-05-30 10:23:32 +02:00
2023-02-28 19:40:57 +01:00
2023-06-06 17:11:30 +01:00
2023-03-21 19:22:01 +01:00
2023-05-24 13:52:52 +01:00
2023-05-18 11:04:51 +01:00
2023-05-23 14:37:35 -04:00
2023-02-28 19:40:57 +01:00
2023-02-28 19:40:57 +01:00
2023-02-28 19:40:57 +01:00
2023-02-28 19:40:57 +01:00
2023-02-28 19:40:57 +01:00
2023-02-28 19:40:57 +01:00
2023-06-06 18:30:51 +01:00
2023-05-24 13:52:52 +01:00
2023-02-28 19:40:57 +01:00
2023-05-24 13:52:52 +01:00
2023-02-28 19:40:57 +01:00
2023-05-24 13:52:52 +01:00
2023-02-28 19:40:57 +01:00
2023-06-12 11:39:04 +02:00
2023-02-28 19:40:57 +01:00
2022-05-03 14:42:02 +02:00
2023-04-17 12:41:55 +02:00
2023-02-28 19:40:57 +01:00
2023-06-06 18:30:51 +01:00
2023-02-28 19:40:57 +01:00
2023-06-13 19:04:22 +01:00
2023-05-24 13:52:52 +01:00
2023-03-06 09:15:44 +01:00
2023-05-24 13:52:52 +01:00
2023-03-29 16:16:23 +02:00
2023-05-24 13:52:52 +01:00
2023-03-29 16:16:23 +02:00
2023-03-07 07:34:04 -05:00
2023-02-28 19:40:57 +01:00
2022-05-03 14:42:02 +02:00