From 5f4f6b65b3beba81ec436b605382a3c14a392c46 Mon Sep 17 00:00:00 2001 From: Manuel Romero Date: Sat, 2 May 2020 17:19:31 +0200 Subject: [PATCH] distilroberta-base-finetuned-sentiment (#4115) * Create model card Create Model card for distilroberta-base-finetuned-sentiment * Update model_cards/mrm8488/distilroberta-base-finetuned-sentiment/README.md * Update model_cards/mrm8488/distilroberta-base-finetuned-sentiment/README.md Co-authored-by: Julien Chaumond --- .../README.md | 65 +++++++++++++++++++ 1 file changed, 65 insertions(+) create mode 100644 model_cards/mrm8488/distilroberta-base-finetuned-sentiment/README.md diff --git a/model_cards/mrm8488/distilroberta-base-finetuned-sentiment/README.md b/model_cards/mrm8488/distilroberta-base-finetuned-sentiment/README.md new file mode 100644 index 0000000000..f1ad6844ad --- /dev/null +++ b/model_cards/mrm8488/distilroberta-base-finetuned-sentiment/README.md @@ -0,0 +1,65 @@ +# DistilRoBERTa + Sentiment Analysis ๐Ÿ˜‚๐Ÿ˜ข๐Ÿ˜ก๐Ÿ˜ƒ๐Ÿ˜ฏ + +This in an adaption version of [@omarsar0](https://twitter.com/omarsar0) [tutorial](https://t.co/WMnATW0Hwf?amp=1) +He explains everything so detailed and provided the dataset. I just changed some parameters and created the ```config.json```file to upload it to [๐Ÿค—Transformers HUB](https://huggingface.co/) + + +In this tutorial, he shows how to fine-tune a language model (LM) for **emotion classification** with code adapted from this [tutorial](https://zablo.net/blog/post/custom-classifier-on-bert-model-guide-polemo2-sentiment-analysis/) by MARCIN ZABลOCKI. + +The emotions covered are: + - sadness ๐Ÿ˜ข + - joy ๐Ÿ˜ƒ + - love ๐Ÿฅฐ + - anger ๐Ÿ˜ก + - fear ๐Ÿ˜ฑ + - surprise ๐Ÿ˜ฏ + +## Details of the language model +The base model used is [DistilRoBERTa](https://huggingface.co/distilroberta-base) + +## Details of the downstream task (Sentence classification) - Dataset ๐Ÿ“š + +| Dataset split | # Size | # Sequences | +| ---------------------- | ----- | ------| +|Train | 1.58M | 20000 +| Validation | 200 KB | +| Test | 202 KB | + + +## Results after training ๐Ÿ‹๏ธโ€โ™€๏ธ๐Ÿงพ + +emotion |precision |recall| f1-score| support| +|-------|-------------|------|----------|----------| +|sadness| 0.973868 |0.949066 |0.961307| 589| +|joy |0.970313 |0.901306 |0.934537| 689| +|love |0.743119 |0.925714 |0.824427| 175| +|anger | 0.884615| 0.969349| 0.925046| 261| +|fear |0.951456 |0.875000| 0.911628| 224| +|surprise| 0.750000| 0.919355| 0.826087| 62| +| | | | | | +|**accuracy**| | | 0.924000| 2000| +|**macro avg**| 0.878895| 0.923298| 0.897172| 2000| +|**weighted avg**| 0.931355| 0.924000| 0.925620| 2000| + +## Model in action ๐Ÿ”จ + +Fast usage with **pipelines** ๐Ÿงช + +```python +from transformers import pipeline + +nlp_sentiment = pipeline( + "sentiment-analysis", + model="mrm8488/distilroberta-base-finetuned-sentiment", + tokenizer="mrm8488/distilroberta-base-finetuned-sentiment" +) + +text = "i feel i should return to the start of the weekend so my loyal readers can get a feeling for things up to this point" + +nlp_sentiment(text) +# Output: [{'label': 'love', 'score': 0.2183746}] +``` + +> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) + +> Made with ♥ in Spain