SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity

[examples] Cleanup summarization docs (#4876)

Browse Source
This commit is contained in:
Sam Shleifer
2020-06-09 17:38:28 -04:00
committed by GitHub
parent 2cfb947f59
commit f90bc44d9a
5 changed files with 4 additions and 62 deletions
Download Patch File Download Diff File Expand all files Collapse all files

28
examples/summarization/README.md
View File

@@ -1,7 +1,4 @@
### Get CNN Data
Both types of models do require CNN data and follow different procedures of obtaining so.
#### For BART models
To be able to reproduce the authors' results on the CNN/Daily Mail dataset you first need to download both CNN and Daily Mail datasets [from Kyunghyun Cho's website](https://cs.nyu.edu/~kcho/DMQA/) (the links next to "Stories") in the same folder. Then uncompress the archives by running:
```bash
@@ -12,40 +9,17 @@ tar -xzvf cnn_dm.tgz
this should make a directory called cnn_dm/ with files like `test.source`.
To use your own data, copy that files format. Each article to be summarized is on its own line.
#### For T5 models
First, you need to download the CNN data. It's about ~400 MB and can be downloaded by
running
```bash
python download_cnn_daily_mail.py cnn_articles_input_data.txt cnn_articles_reference_summaries.txt
```
You should confirm that each file has 11490 lines:
```bash
wc -l cnn_articles_input_data.txt # should print 11490
wc -l cnn_articles_reference_summaries.txt # should print 11490
```
### Evaluation
To create summaries for each article in dataset, run:
```bash
python evaluate_cnn.py <path_to_test.source> test_generations.txt <model-name>
python evaluate_cnn.py <path_to_test.source> test_generations.txt <model-name> --score_path rouge_scores.txt
```
The default batch size, 8, fits in 16GB GPU memory, but may need to be adjusted to fit your system.
### Training
Run/modify `finetune_bart.sh` or `finetune_t5.sh`
## (WIP) Rouge Scores
To create summaries for each article in dataset and also calculate rouge scores run:
```bash
python evaluate_cnn.py <path_to_test.source> test_generations.txt <model-name> --reference_path <path_to_correct_summaries> --score_path <path_to_save_rouge_scores>
```
The rouge scores "rouge1, rouge2, rougeL" are automatically created and saved in ``<path_to_save_rouge_scores>``.
### Stanford CoreNLP Setup
```
ptb_tokenize () {