From d31031f603043281d4fbac6cbdcfb6497fd500ab Mon Sep 17 00:00:00 2001 From: Steven Liu <59462357+stevhliu@users.noreply.github.com> Date: Sat, 5 Sep 2020 09:50:19 -0700 Subject: [PATCH] create model card for astroGPT (#6960) * create model card for astroGPT * Hotlink to actual image file Co-authored-by: Julien Chaumond --- model_cards/stevhliu/astroGPT/README.md | 51 +++++++++++++++++++++++++ 1 file changed, 51 insertions(+) create mode 100644 model_cards/stevhliu/astroGPT/README.md diff --git a/model_cards/stevhliu/astroGPT/README.md b/model_cards/stevhliu/astroGPT/README.md new file mode 100644 index 0000000000..f2f2a9e510 --- /dev/null +++ b/model_cards/stevhliu/astroGPT/README.md @@ -0,0 +1,51 @@ +--- +language: "en" +thumbnail: "https://raw.githubusercontent.com/stevhliu/satsuma/master/images/astroGPT-thumbnail.png" +widget: +- text: "Jan 18, 2020" +- text: "Feb 14, 2020" +- text: "Jul 04, 2020" +--- + +# astroGPT 🪐 + +## Model description + +This is a GPT-2 model fine-tuned on Western zodiac signs. For more information about GPT-2, take a look at 🤗 Hugging Face's GPT-2 [model card](https://huggingface.co/gpt2). You can use astroGPT to generate a daily horoscope by entering the current date. + +## How to use + +To use this model, simply enter the current date like so `Mon DD, YEAR`: + +```python +from transformers import AutoTokenizer, AutoModelWithLMHead + +tokenizer = AutoTokenizer.from_pretrained("stevhliu/astroGPT") +model = AutoModelWithLMHead.from_pretrained("stevhliu/astroGPT") + +input_ids = tokenizer.encode('Sep 03, 2020', return_tensors='pt').to('cuda') + +sample_output = model.generate(input_ids, + do_sample=True, + max_length=75, + top_k=20, + top_p=0.97) + +print(sample_output) +``` + +## Limitations and bias + +astroGPT inherits the same biases that affect GPT-2 as a result of training on a lot of non-neutral content on the internet. The model does not currently support zodiac sign-specific generation and only returns a general horoscope. While the generated text may occasionally mention a specific zodiac sign, this is due to how the horoscopes were originally written by it's human authors. + +## Data + +The data was scraped from [Horoscope.com](https://www.horoscope.com/us/index.aspx) and trained on 4.7MB of text. The text was collected from four categories (daily, love, wellness, career) and span from 09/01/19 to 08/01/2020. The archives only store horoscopes dating a year back from the current date. + +## Training and results + +The text was tokenized using the fast GPT-2 BPE [tokenizer](https://huggingface.co/transformers/model_doc/gpt2.html#gpt2tokenizerfast). It has a vocabulary size of 50,257 and sequence length of 1024 tokens. The model was trained with on one of Google Colaboratory's GPU's for approximately 2.5 hrs with [fastai's](https://docs.fast.ai/) learning rate finder, discriminative learning rates and 1cycle policy. See table below for a quick summary of the training procedure and results. + +| dataset size | epochs | lr | training time | train_loss | valid_loss | perplexity | +|:-------------:|:------:|:-----------------:|:-------------:|:----------:|:----------:|:----------:| +| 5.9MB |32 | slice(1e-7,1e-5) | 2.5 hrs | 2.657170 | 2.642387 | 14.046692 |