SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity

Fix many typos (#8708)

Browse Source
This commit is contained in:
Santiago Castro
2020-11-22 00:58:10 -03:00
committed by GitHub
parent 9c0afdaf7b
commit e1f3156b21
35 changed files with 51 additions and 51 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
docs/source/model_summary.rst
View File

@@ -530,7 +530,7 @@ Sequence-to-sequence model with the same encoder-decoder model architecture as B
two self-supervised objective functions: Masked Language Modeling (MLM) and a novel summarization specific pre-training
objective, called Gap Sentence Generation (GSG).
* MLM: encoder input tokens are randomely replaced by a mask tokens and have to be predicted by the encoder (like in
* MLM: encoder input tokens are randomly replaced by a mask tokens and have to be predicted by the encoder (like in
BERT)
* GSG: whole encoder input sentences are replaced by a second mask token and fed to the decoder, but which has a
causal mask to hide the future words like a regular auto-regressive transformer decoder.