SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity

Upload DistilBART artwork (#5394)

Browse Source
This commit is contained in:
Kevin Canwen Xu
2020-06-30 18:11:11 +08:00
committed by GitHub
parent 09e841490c
commit 331d8d2936
1 changed files with 3 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
examples/seq2seq/README.md
View File

@@ -1,3 +1,5 @@
## Sequence to Sequence
This directory contains examples for finetuning and evaluating transformers on summarization and translation tasks.
Summarization support is more mature than translation support.
Please tag @sshleifer with any issues/unexpected behaviors, or send a PR!
@@ -168,6 +170,7 @@ python run_eval.py sshleifer/distilbart-cnn-12-6 $DATA_DIR/val.source dbart_val_
### DistilBART
![DBART](https://huggingface.co/front/thumbnails/distilbart_large.png)
For the CNN/DailyMail dataset, (relatively longer, more extractive summaries), we found a simple technique that works:
you just copy alternating layers from `bart-large-cnn` and finetune more on the same data.