From 6083c1566e261668a5de73cfe484c171ce232812 Mon Sep 17 00:00:00 2001 From: Manuel Romero Date: Sun, 16 Feb 2020 13:10:53 +0100 Subject: [PATCH] Update README.md I trained the model for more epochs so I improved the results. This commit will update the results of the model and add a gif using it with **transformers/pipelines** --- .../README.md | 28 +++++++++++-------- 1 file changed, 16 insertions(+), 12 deletions(-) diff --git a/model_cards/mrm8488/distill-bert-base-spanish-wwm-cased-finetuned-spa-squad2-es/README.md b/model_cards/mrm8488/distill-bert-base-spanish-wwm-cased-finetuned-spa-squad2-es/README.md index 24bd203cf6..001d1bdd13 100644 --- a/model_cards/mrm8488/distill-bert-base-spanish-wwm-cased-finetuned-spa-squad2-es/README.md +++ b/model_cards/mrm8488/distill-bert-base-spanish-wwm-cased-finetuned-spa-squad2-es/README.md @@ -46,7 +46,7 @@ The model was trained on a Tesla P100 GPU and 25GB of RAM with the following com --predict_file $SQUAD_DIR/dev-v2.json \ --per_gpu_train_batch_size 12 \ --learning_rate 3e-5 \ - --num_train_epochs 3.0 \ + --num_train_epochs 5.0 \ --max_seq_length 384 \ --doc_stride 128 \ --output_dir /content/model_output \ @@ -59,23 +59,23 @@ The model was trained on a Tesla P100 GPU and 25GB of RAM with the following com | Metric | # Value | | --------- | ----------- | -| **Exact** | **82.40**65 | -| **F1** | **90.50**36 | +| **Exact** | **90.77**48 | +| **F1** | **94.94**71 | ```json { - "exact": 82.40657784457096, - "f1": 90.50369643376753, + "exact": 90.77483309730933, + "f1": 94.94714391266254, "total": 69202, - "HasAns_exact": 75.13413304253, - "HasAns_f1": 87.35521920631561, + "HasAns_exact": 86.60850599781898, + "HasAns_f1": 92.90582885592328, "HasAns_total": 45850, - "NoAns_exact": 96.68550873586845, - "NoAns_f1": 96.68550873586845, + "NoAns_exact": 98.95512161699212, + "NoAns_f1": 98.95512161699212, "NoAns_total": 23352, - "best_exact": 82.40657784457096, + "best_exact": 90.77483309730933, "best_exact_thresh": 0.0, - "best_f1": 90.50369643376902, + "best_f1": 94.94714391266305, "best_f1_thresh": 0.0 } ``` @@ -85,7 +85,7 @@ The model was trained on a Tesla P100 GPU and 25GB of RAM with the following com | Model | f1 score | | :-------------------------------------------------------------: | :-------: | | bert-base-spanish-wwm-cased-finetuned-spa-squad2-es | 86.07 | -| **distill**-bert-base-spanish-wwm-cased-finetuned-spa-squad2-es | **90.50** | +| **distill**-bert-base-spanish-wwm-cased-finetuned-spa-squad2-es | **94.94** | So, yes, this version is even more accurate. @@ -101,6 +101,10 @@ So, yes, this version is even more accurate. ![Run the model](https://media.giphy.com/media/WT453aptcbCP7hxWTZ/giphy.gif) +3. Using **Pipelines** + +![qa-pipelines](https://media.giphy.com/media/L4YiWhinC3P1GF31t0/giphy.gif) + > Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488)