Update documentation on seq2seq models with absolute positional embeddings, to be in line with Tips section for BERT and GPT2 (#20068)

Co-authored-by: jordiclive <jordiclive19@imperial.ac.uk>
This commit is contained in:
Jordan Clive
2022-11-04 15:32:44 +00:00
committed by GitHub
parent 6e1c5786dc
commit 3bd0007e87
7 changed files with 26 additions and 0 deletions

View File

@@ -35,6 +35,11 @@ dataset (160GB) respectively. Then we conduct experiments on CNN/DailyMail, Giga
abstractive summarization and question generation tasks. Experimental results show that ProphetNet achieves new
state-of-the-art results on all these datasets compared to the models using the same scale pretraining corpus.*
Tips:
- ProphetNet is a model with absolute position embeddings so it's usually advised to pad the inputs on the right rather than
the left.
The Authors' code can be found [here](https://github.com/microsoft/ProphetNet).