SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity

Upload DistilBART artwork (#5394)

Browse Source
This commit is contained in:
Kevin Canwen Xu
2020-06-30 18:11:11 +08:00
committed by GitHub
parent 09e841490c
commit 331d8d2936
1 changed files with 3 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
examples/seq2seq/README.md
View File

@@ -1,3 +1,5 @@
## Sequence to Sequence
This directory contains examples for finetuning and evaluating transformers on summarization and translation tasks. This directory contains examples for finetuning and evaluating transformers on summarization and translation tasks.
Summarization support is more mature than translation support. Summarization support is more mature than translation support.
Please tag @sshleifer with any issues/unexpected behaviors, or send a PR! Please tag @sshleifer with any issues/unexpected behaviors, or send a PR!
@@ -168,6 +170,7 @@ python run_eval.py sshleifer/distilbart-cnn-12-6 $DATA_DIR/val.source dbart_val_
### DistilBART ### DistilBART
![DBART](https://huggingface.co/front/thumbnails/distilbart_large.png)
For the CNN/DailyMail dataset, (relatively longer, more extractive summaries), we found a simple technique that works: For the CNN/DailyMail dataset, (relatively longer, more extractive summaries), we found a simple technique that works:
you just copy alternating layers from `bart-large-cnn` and finetune more on the same data. you just copy alternating layers from `bart-large-cnn` and finetune more on the same data.