Eli Simhayev
4b6a5a7caa
[Time-Series] Autoformer model (#21891)
* ran `transformers-cli add-new-model-like`
* added `AutoformerLayernorm` and `AutoformerSeriesDecomposition`
* added `decomposition_layer` in `init` and `moving_avg` to config
* added `AutoformerAutoCorrelation` to encoder & decoder
* removed caninical self attention `AutoformerAttention`
* added arguments in config and model tester. Init works! 😁
* WIP autoformer attention with autocorrlation
* fixed `attn_weights` size
* wip time_delay_agg_training
* fixing sizes and debug time_delay_agg_training
* aggregation in training works! 😁
* `top_k_delays` -> `top_k_delays_index` and added `contiguous()`
* wip time_delay_agg_inference
* finish time_delay_agg_inference 😎
* added resize to autocorrelation
* bug fix: added the length of the output signal to `irfft`
* `attention_mask = None` in the decoder
* fixed test: changed attention expected size, `test_attention_outputs` works!
* removed unnecessary code
* apply AutoformerLayernorm in final norm in enc & dec
* added series decomposition to the encoder
* added series decomp to decoder, with inputs
* added trend todos
* added autoformer to README
* added to index
* added autoformer.mdx
* remove scaling and init attention_mask in the decoder
* make style
* fix copies
* make fix-copies
* inital fix-copies
* fix from https://github.com/huggingface/transformers/pull/22076
* make style
* fix class names
* added trend
* added d_model and projection layers
* added `trend_projection` source, and decomp layer init
* added trend & seasonal init for decoder input
* AutoformerModel cannot be copied as it has the decomp layer too
* encoder can be copied from time series transformer
* fixed generation and made distrb. out more robust
* use context window to calculate decomposition
* use the context_window for decomposition
* use output_params helper
* clean up AutoformerAttention
* subsequences_length off by 1
* make fix copies
* fix test
* added init for nn.Conv1d
* fix IGNORE_NON_TESTED
* added model_doc
* fix ruff
* ignore tests
* remove dup
* fix SPECIAL_CASES_TO_ALLOW
* do not copy due to conv1d weight init
* remove unused imports
* added short summary
* added label_length and made the model non-autoregressive
* added params docs
* better doc for `factor`
* fix tests
* renamed `moving_avg` to `moving_average`
* renamed `factor` to `autocorrelation_factor`
* make style
* Update src/transformers/models/autoformer/configuration_autoformer.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update src/transformers/models/autoformer/configuration_autoformer.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* fix configurations
* fix integration tests
* Update src/transformers/models/autoformer/configuration_autoformer.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* fixing `lags_sequence` doc
* Revert "fixing `lags_sequence` doc"
This reverts commit 21e34911e36a6f8f45f25cbf43584a49e5316c55.
* Update src/transformers/models/autoformer/modeling_autoformer.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/autoformer/modeling_autoformer.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/autoformer/modeling_autoformer.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/autoformer/configuration_autoformer.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* model layers now take the config
* added `layer_norm_eps` to the config
* Update src/transformers/models/autoformer/modeling_autoformer.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* added `config.layer_norm_eps` to AutoformerLayernorm
* added `config.layer_norm_eps` to all layernorm layers
* Update src/transformers/models/autoformer/configuration_autoformer.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/autoformer/configuration_autoformer.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/autoformer/configuration_autoformer.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/autoformer/configuration_autoformer.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* fix variable names
* added inital pretrained model
* added use_cache docstring
* doc strings for trend and use_cache
* fix order of args
* imports on one line
* fixed get_lagged_subsequences docs
* add docstring for create_network_inputs
* get rid of layer_norm_eps config
* add back layernorm
* update fixture location
* fix signature
* use AutoformerModelOutput dataclass
* fix pretrain config
* no need as default exists
* subclass ModelOutput
* remove layer_norm_eps config
* fix test_model_outputs_equivalence test
* test hidden_states_output
* make fix-copies
* Update src/transformers/models/autoformer/configuration_autoformer.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* removed unused attr
* Update tests/models/autoformer/test_modeling_autoformer.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/autoformer/modeling_autoformer.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/autoformer/modeling_autoformer.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/autoformer/modeling_autoformer.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/autoformer/modeling_autoformer.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/autoformer/modeling_autoformer.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/autoformer/modeling_autoformer.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* use AutoFormerDecoderOutput
* fix formatting
* fix formatting
---------
Co-authored-by: Kashif Rasul <kashif.rasul@gmail.com>
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
2023-05-30 10:23:32 +02:00
..
2023-05-24 13:52:52 +01:00
2023-05-15 16:46:58 +02:00
2023-03-23 19:14:17 +01:00
2023-05-23 14:37:35 -04:00
2023-05-24 13:52:52 +01:00
2023-05-30 10:23:32 +02:00
2023-05-24 13:52:52 +01:00
2023-02-06 18:10:56 -05:00
2023-02-06 18:10:56 -05:00
2023-02-28 19:40:57 +01:00
2023-05-24 13:52:52 +01:00
2023-02-28 19:40:57 +01:00
2023-02-06 18:10:56 -05:00
2023-02-06 18:10:56 -05:00
2023-05-07 18:55:04 -04:00
2023-05-18 11:04:51 +01:00
2023-05-18 17:29:04 +02:00
2023-04-06 13:50:15 +01:00
2023-05-24 13:52:52 +01:00
2023-05-24 13:52:52 +01:00
2023-05-24 13:52:52 +01:00
2023-03-13 15:03:21 +01:00
2023-05-16 19:28:19 +01:00
2023-05-24 13:52:52 +01:00
2023-05-23 17:32:57 +02:00
2023-02-06 18:10:56 -05:00
2023-05-24 13:52:52 +01:00
2023-02-28 19:40:57 +01:00
2023-02-28 19:40:57 +01:00
2023-05-23 14:37:35 -04:00
2023-05-24 13:52:52 +01:00
2023-02-28 19:40:57 +01:00
2023-04-24 15:27:31 +02:00
2023-03-07 14:23:36 +01:00
2023-05-24 13:52:52 +01:00
2023-05-24 13:52:52 +01:00
2023-05-16 23:35:11 +02:00
2023-03-22 20:02:24 +01:00
2023-05-18 17:29:04 +02:00
2023-05-24 13:52:52 +01:00
2023-05-24 13:52:52 +01:00
2023-05-24 13:52:52 +01:00
2023-05-24 13:52:52 +01:00
2023-05-24 13:52:52 +01:00
2023-02-28 19:40:57 +01:00
2023-02-28 19:40:57 +01:00
2023-05-24 13:52:52 +01:00
2023-04-13 21:32:29 +02:00
2023-03-07 15:19:39 -05:00
2023-04-06 13:50:15 +01:00
2023-05-24 13:52:52 +01:00
2022-05-03 14:42:02 +02:00
2023-02-28 19:40:57 +01:00
2023-05-24 13:52:52 +01:00
2023-02-28 19:40:57 +01:00
2023-02-28 19:40:57 +01:00
2023-04-06 22:52:59 +02:00
2023-05-24 13:52:52 +01:00
2023-05-24 13:52:52 +01:00
2023-02-28 19:40:57 +01:00
2023-03-23 19:14:17 +01:00
2023-05-24 13:52:52 +01:00
2023-05-24 13:52:52 +01:00
2023-02-28 19:40:57 +01:00
2023-04-19 20:27:19 +02:00
2023-05-18 17:29:04 +02:00
2023-04-06 17:56:06 +02:00
2023-05-24 13:52:52 +01:00
2023-02-28 19:40:57 +01:00
2023-02-28 19:40:57 +01:00
2023-05-24 13:52:52 +01:00
2023-05-16 19:28:19 +01:00
2023-05-18 17:29:04 +02:00
2023-05-18 17:29:04 +02:00
2023-02-28 19:40:57 +01:00
2022-12-12 13:12:13 -05:00
2023-05-24 13:52:52 +01:00
2023-04-06 22:52:59 +02:00
2023-02-28 19:40:57 +01:00
2023-05-24 13:52:52 +01:00
2023-02-06 18:10:56 -05:00
2023-05-24 13:52:52 +01:00
2023-02-28 19:40:57 +01:00
2023-05-18 11:04:51 +01:00
2023-05-30 10:23:32 +02:00
2023-03-07 04:20:14 +01:00
2023-05-24 13:52:52 +01:00
2023-04-17 17:45:20 +02:00
2023-05-24 13:52:52 +01:00
2023-02-22 09:14:54 +01:00
2023-05-24 13:52:52 +01:00
2023-02-28 19:40:57 +01:00
2023-03-14 10:03:02 +01:00
2023-04-06 09:53:03 +02:00
2023-05-24 13:52:52 +01:00
2023-04-06 17:56:06 +02:00
2023-03-22 20:02:24 +01:00
2023-05-24 13:52:52 +01:00
2023-04-06 17:56:06 +02:00
2023-05-24 13:52:52 +01:00
2023-04-11 10:00:34 +02:00
2023-03-22 20:45:08 -04:00
2023-04-06 13:50:15 +01:00
2023-05-24 13:52:52 +01:00
2022-07-29 08:09:09 -04:00
2023-05-23 14:37:35 -04:00
2023-04-07 17:13:04 +02:00
2023-02-28 19:40:57 +01:00
2022-05-03 14:42:02 +02:00
2023-04-06 17:56:06 +02:00
2022-05-12 16:25:55 -04:00
2023-05-24 13:52:52 +01:00
2023-02-28 19:40:57 +01:00
2023-02-28 19:40:57 +01:00
2023-05-24 13:52:52 +01:00
2023-05-24 13:52:52 +01:00
2023-05-24 13:52:52 +01:00
2023-05-18 11:04:51 +01:00
2023-04-06 13:50:15 +01:00
2023-02-28 19:40:57 +01:00
2023-04-04 14:53:06 +02:00
2023-04-21 17:09:40 +01:00
2023-02-28 19:40:57 +01:00
2023-03-23 19:14:17 +01:00
2023-04-28 11:01:32 -04:00
2023-05-24 13:52:52 +01:00
2023-05-24 13:52:52 +01:00
2023-02-28 19:40:57 +01:00
2023-05-24 13:52:52 +01:00
2023-04-06 17:56:06 +02:00
2023-02-28 19:40:57 +01:00
2023-02-06 18:10:56 -05:00
2023-05-16 14:49:24 +02:00
2023-05-18 11:04:51 +01:00
2023-02-28 19:40:57 +01:00
2023-05-18 11:04:51 +01:00
2023-02-28 19:40:57 +01:00
2023-05-24 13:52:52 +01:00
2023-02-28 19:40:57 +01:00
2023-05-18 11:04:51 +01:00
2023-05-24 13:52:52 +01:00
2023-05-24 13:52:52 +01:00
2023-05-24 13:52:52 +01:00
2023-02-06 18:10:56 -05:00
2023-05-24 13:52:52 +01:00
2023-05-24 13:52:52 +01:00
2023-03-14 10:03:02 +01:00
2023-05-24 13:52:52 +01:00
2023-05-18 17:29:04 +02:00
2023-05-24 13:52:52 +01:00
2023-05-24 13:52:52 +01:00
2023-02-28 19:40:57 +01:00
2023-02-28 19:40:57 +01:00
2023-02-06 18:10:56 -05:00
2023-05-24 13:52:52 +01:00
2023-02-28 19:40:57 +01:00
2023-05-23 14:37:35 -04:00
2023-04-06 17:45:55 +02:00
2023-02-28 19:40:57 +01:00
2023-05-18 17:29:04 +02:00
2023-05-24 13:52:52 +01:00
2023-03-07 14:23:36 +01:00
2023-03-07 14:23:36 +01:00
2023-04-06 17:56:06 +02:00
2023-05-24 13:52:52 +01:00
2023-02-28 19:40:57 +01:00
2023-05-24 13:52:52 +01:00
2023-02-06 18:10:56 -05:00
2023-05-30 10:23:32 +02:00
2023-02-28 19:40:57 +01:00
2023-03-21 19:22:01 +01:00
2023-05-24 13:52:52 +01:00
2023-05-18 11:04:51 +01:00
2023-05-23 14:37:35 -04:00
2023-02-28 19:40:57 +01:00
2023-02-28 19:40:57 +01:00
2023-02-28 19:40:57 +01:00
2023-02-28 19:40:57 +01:00
2023-02-28 19:40:57 +01:00
2023-02-28 19:40:57 +01:00
2023-05-24 13:52:52 +01:00
2023-05-24 13:52:52 +01:00
2023-02-28 19:40:57 +01:00
2023-05-24 13:52:52 +01:00
2023-02-28 19:40:57 +01:00
2023-05-24 13:52:52 +01:00
2023-02-28 19:40:57 +01:00
2023-05-24 13:52:52 +01:00
2023-02-28 19:40:57 +01:00
2022-05-03 14:42:02 +02:00
2023-04-17 12:41:55 +02:00
2023-02-28 19:40:57 +01:00
2023-05-24 17:31:25 +01:00
2023-02-28 19:40:57 +01:00
2023-05-24 13:52:52 +01:00
2023-05-24 13:52:52 +01:00
2023-03-06 09:15:44 +01:00
2023-05-24 13:52:52 +01:00
2023-03-29 16:16:23 +02:00
2023-05-24 13:52:52 +01:00
2023-03-29 16:16:23 +02:00
2023-03-07 07:34:04 -05:00
2023-02-28 19:40:57 +01:00
2022-05-03 14:42:02 +02:00