From 4a5663568f562b15ccc4ea70e027ae9c07ac978e Mon Sep 17 00:00:00 2001 From: Manuel Romero Date: Tue, 31 Mar 2020 14:01:03 +0200 Subject: [PATCH] Create card for the model: GPT-2-finetuned-covid-bio-medrxiv (#3453) --- .../README.md | 62 +++++++++++++++++++ 1 file changed, 62 insertions(+) create mode 100644 model_cards/mrm8488/GPT-2-finetuned-covid-bio-medrxiv/README.md diff --git a/model_cards/mrm8488/GPT-2-finetuned-covid-bio-medrxiv/README.md b/model_cards/mrm8488/GPT-2-finetuned-covid-bio-medrxiv/README.md new file mode 100644 index 0000000000..cb0251c5a6 --- /dev/null +++ b/model_cards/mrm8488/GPT-2-finetuned-covid-bio-medrxiv/README.md @@ -0,0 +1,62 @@ +--- +language: english +thumbnail: +--- + +# GPT-2 + bio/medrxiv files from CORD19: 🦠 ✍ ⚕ + +**GPT-2** fine-tuned on **biorxiv_medrxiv** files from [CORD-19](https://www.kaggle.com/allen-institute-for-ai/CORD-19-research-challenge) dataset. + + +## Datasets details: + +| Dataset | # Files | +| ---------------------- | ----- | +| biorxiv_medrxiv | 885 | + + +## Model training: + +The model was trained on a Tesla P100 GPU and 25GB of RAM with the following command: + +```bash + +export TRAIN_FILE=/path/to/dataset/train.txt + +python run_language_modeling.py \ + --model_type gpt2 \ + --model_name_or_path gpt2 \ + --do_train \ + --train_data_file $TRAIN_FILE \ + --num_train_epochs 4 \ + --output_dir model_output \ + --overwrite_output_dir \ + --save_steps 2000 \ + --per_gpu_train_batch_size 3 +``` + +## Model in action / Example of usage: ✒ + +You can get the following script [here](https://github.com/huggingface/transformers/blob/master/examples/run_generation.py) + +```bash +python run_generation.py \ + --model_type gpt2 \ + --model_name_or_path mrm8488/GPT-2-finetuned-CORD19 \ + --length 200 +``` +```txt +👵👴🦠 +# Input: Old people with COVID-19 tends to suffer +# Output: === GENERATED SEQUENCE 1 === +Old people with COVID-19 tends to suffer more symptom onset time and death. It is well known that many people with COVID-19 have high homozygous ZIKV infection in the face of severe symptoms in both severe and severe cases. +The origin of Wuhan Fever was investigated by Prof. Shen Jiang at the outbreak of Wuhan Fever [34]. As Huanan Province is the epicenter of this outbreak, Huanan, the epicenter of epidemic Wuhan Fever, is the most potential location for the direct transmission of infection (source: Zhongzhen et al., 2020). A negative risk ratio indicates more frequent underlying signs in the people in Huanan Province with COVID-19 patients. Further analysis of reported Huanan Fever onset data in the past two years indicated that the intensity of exposure is the key risk factor for developing MERS-CoV infection in this region, especially among children and elderly. To be continued to develop infected patients would be a very important area for +``` + +![Model in action](https://media.giphy.com/media/TgUdO72Iwk9h7hhm7G/giphy.gif) + + + +> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) | [LinkedIn](https://www.linkedin.com/in/manuel-romero-cs/) + +> Made with ♥ in Spain