Eli Simhayev
4b6a5a7caa
[Time-Series] Autoformer model (#21891)
* ran `transformers-cli add-new-model-like`
* added `AutoformerLayernorm` and `AutoformerSeriesDecomposition`
* added `decomposition_layer` in `init` and `moving_avg` to config
* added `AutoformerAutoCorrelation` to encoder & decoder
* removed caninical self attention `AutoformerAttention`
* added arguments in config and model tester. Init works! 😁
* WIP autoformer attention with autocorrlation
* fixed `attn_weights` size
* wip time_delay_agg_training
* fixing sizes and debug time_delay_agg_training
* aggregation in training works! 😁
* `top_k_delays` -> `top_k_delays_index` and added `contiguous()`
* wip time_delay_agg_inference
* finish time_delay_agg_inference 😎
* added resize to autocorrelation
* bug fix: added the length of the output signal to `irfft`
* `attention_mask = None` in the decoder
* fixed test: changed attention expected size, `test_attention_outputs` works!
* removed unnecessary code
* apply AutoformerLayernorm in final norm in enc & dec
* added series decomposition to the encoder
* added series decomp to decoder, with inputs
* added trend todos
* added autoformer to README
* added to index
* added autoformer.mdx
* remove scaling and init attention_mask in the decoder
* make style
* fix copies
* make fix-copies
* inital fix-copies
* fix from https://github.com/huggingface/transformers/pull/22076
* make style
* fix class names
* added trend
* added d_model and projection layers
* added `trend_projection` source, and decomp layer init
* added trend & seasonal init for decoder input
* AutoformerModel cannot be copied as it has the decomp layer too
* encoder can be copied from time series transformer
* fixed generation and made distrb. out more robust
* use context window to calculate decomposition
* use the context_window for decomposition
* use output_params helper
* clean up AutoformerAttention
* subsequences_length off by 1
* make fix copies
* fix test
* added init for nn.Conv1d
* fix IGNORE_NON_TESTED
* added model_doc
* fix ruff
* ignore tests
* remove dup
* fix SPECIAL_CASES_TO_ALLOW
* do not copy due to conv1d weight init
* remove unused imports
* added short summary
* added label_length and made the model non-autoregressive
* added params docs
* better doc for `factor`
* fix tests
* renamed `moving_avg` to `moving_average`
* renamed `factor` to `autocorrelation_factor`
* make style
* Update src/transformers/models/autoformer/configuration_autoformer.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update src/transformers/models/autoformer/configuration_autoformer.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* fix configurations
* fix integration tests
* Update src/transformers/models/autoformer/configuration_autoformer.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* fixing `lags_sequence` doc
* Revert "fixing `lags_sequence` doc"
This reverts commit 21e34911e36a6f8f45f25cbf43584a49e5316c55.
* Update src/transformers/models/autoformer/modeling_autoformer.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/autoformer/modeling_autoformer.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/autoformer/modeling_autoformer.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/autoformer/configuration_autoformer.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* model layers now take the config
* added `layer_norm_eps` to the config
* Update src/transformers/models/autoformer/modeling_autoformer.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* added `config.layer_norm_eps` to AutoformerLayernorm
* added `config.layer_norm_eps` to all layernorm layers
* Update src/transformers/models/autoformer/configuration_autoformer.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/autoformer/configuration_autoformer.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/autoformer/configuration_autoformer.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/autoformer/configuration_autoformer.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* fix variable names
* added inital pretrained model
* added use_cache docstring
* doc strings for trend and use_cache
* fix order of args
* imports on one line
* fixed get_lagged_subsequences docs
* add docstring for create_network_inputs
* get rid of layer_norm_eps config
* add back layernorm
* update fixture location
* fix signature
* use AutoformerModelOutput dataclass
* fix pretrain config
* no need as default exists
* subclass ModelOutput
* remove layer_norm_eps config
* fix test_model_outputs_equivalence test
* test hidden_states_output
* make fix-copies
* Update src/transformers/models/autoformer/configuration_autoformer.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* removed unused attr
* Update tests/models/autoformer/test_modeling_autoformer.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/autoformer/modeling_autoformer.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/autoformer/modeling_autoformer.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/autoformer/modeling_autoformer.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/autoformer/modeling_autoformer.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/autoformer/modeling_autoformer.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/autoformer/modeling_autoformer.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* use AutoFormerDecoderOutput
* fix formatting
* fix formatting
---------
Co-authored-by: Kashif Rasul <kashif.rasul@gmail.com>
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
2023-05-30 10:23:32 +02:00
..
2023-03-17 14:30:17 +00:00
2023-03-16 13:41:48 +03:00
2023-01-04 09:18:57 +01:00
2023-03-17 14:30:17 +00:00
2023-04-21 10:04:45 -04:00
2023-05-30 10:23:32 +02:00
2023-03-17 14:30:17 +00:00
2022-04-04 10:25:46 -04:00
2022-04-04 10:25:46 -04:00
2023-03-17 14:30:17 +00:00
2022-04-29 17:42:15 -04:00
2022-04-04 10:25:46 -04:00
2023-03-17 14:30:17 +00:00
2022-04-04 10:25:46 -04:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-05-01 09:17:27 -04:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-02-28 15:42:55 +01:00
2023-04-04 16:05:22 +01:00
2023-03-17 14:30:17 +00:00
2022-04-04 10:25:46 -04:00
2023-03-08 09:00:54 -05:00
2022-04-13 11:36:54 +02:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2022-11-30 19:22:23 +01:00
2023-04-24 14:00:29 +02:00
2023-01-17 17:18:56 +01:00
2022-11-09 18:31:22 +01:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-03-14 12:08:14 +03:00
2022-05-02 12:47:39 -04:00
2023-04-12 07:33:20 -04:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2022-04-04 10:25:46 -04:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-04-05 17:43:48 +02:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2022-04-04 10:25:46 -04:00
2023-03-17 14:30:17 +00:00
2023-05-03 18:23:09 +02:00
2023-01-17 17:18:56 +01:00
2022-11-29 10:38:01 +00:00
2023-02-15 10:35:14 -08:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-02-20 16:37:11 +03:00
2023-03-17 14:30:17 +00:00
2022-07-27 10:08:59 +02:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2022-12-27 02:26:14 -05:00
2023-03-10 07:44:45 -05:00
2023-03-17 14:30:17 +00:00
2022-11-08 19:54:41 +00:00
2023-03-17 14:30:17 +00:00
2023-04-23 20:03:05 +03:00
2022-04-04 10:25:46 -04:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-03-17 10:27:12 -04:00
2023-05-03 18:23:09 +02:00
2023-04-10 10:57:21 +02:00
2023-05-03 15:59:19 -04:00
2023-03-17 14:30:17 +00:00
2023-05-04 10:15:15 -04:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-02-20 11:25:27 +01:00
2023-05-15 12:04:16 +01:00
2023-01-17 17:18:56 +01:00
2022-04-04 10:25:46 -04:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-03-07 21:36:38 +01:00
2023-02-27 08:36:36 +01:00
2023-03-17 14:30:17 +00:00
2023-05-03 09:53:00 -04:00
2023-03-17 14:30:17 +00:00
2022-04-04 10:25:46 -04:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-05-07 18:52:44 -04:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-01-25 12:34:43 +01:00
2023-01-16 20:37:07 +03:00
2023-04-05 17:43:48 +02:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-04-25 17:58:45 -04:00
2022-04-04 10:25:46 -04:00
2023-03-17 14:30:17 +00:00
2023-03-17 09:40:06 +00:00
2022-04-04 10:25:46 -04:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-04-21 17:09:40 +01:00
2023-04-04 14:53:06 +02:00
2023-03-17 14:30:17 +00:00
2023-01-25 12:34:43 +01:00
2023-04-28 11:01:32 -04:00
2023-03-17 14:30:17 +00:00
2023-05-26 14:30:32 +02:00
2023-01-03 19:25:09 +03:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2022-04-04 10:25:46 -04:00
2023-05-16 14:49:24 +02:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-02-15 10:35:14 -08:00
2022-04-04 10:25:46 -04:00
2023-03-17 14:30:17 +00:00
2023-04-04 12:41:12 -04:00
2023-03-17 14:30:17 +00:00
2023-03-24 19:45:57 +00:00
2022-04-04 10:25:46 -04:00
2023-03-17 14:30:17 +00:00
2023-05-03 18:23:09 +02:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-05-09 13:04:10 -04:00
2023-05-19 14:14:13 +01:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2022-11-03 14:18:45 +01:00
2022-07-29 08:09:09 -04:00
2023-02-03 12:43:46 -05:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-05-12 11:52:31 +01:00
2022-12-16 16:24:01 +01:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-05-03 18:23:09 +02:00
2022-11-07 09:19:04 -05:00
2023-02-21 10:35:11 -05:00
2023-03-17 14:30:17 +00:00
2022-04-08 10:57:51 +02:00
2023-02-13 10:11:16 -05:00
2023-03-17 14:30:17 +00:00
2022-05-17 19:07:43 -04:00
2023-03-17 14:30:17 +00:00
2023-05-05 11:29:20 -04:00
2023-02-15 18:10:30 +00:00
2022-06-21 10:24:50 +02:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2022-11-08 19:54:41 +00:00
2022-11-30 14:50:55 +00:00
2023-03-01 18:00:48 +00:00
2022-11-30 14:50:55 +00:00
2023-03-17 14:30:17 +00:00
2023-01-17 17:18:56 +01:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2022-04-04 10:25:46 -04:00
2023-03-17 14:30:17 +00:00
2023-04-26 13:35:30 +01:00
2023-03-17 14:30:17 +00:00
2023-05-05 13:23:46 -04:00
2023-01-17 17:18:56 +01:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-02-07 16:43:19 -05:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2022-04-04 10:25:46 -04:00
2022-04-04 10:25:46 -04:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00
2023-03-17 14:30:17 +00:00