From f5d69c75f7955f5f5d99b89494253957d0e5c5ce Mon Sep 17 00:00:00 2001 From: Igli Manaj Date: Tue, 1 Sep 2020 23:56:19 +0200 Subject: [PATCH] Update multilingual passage rereanking model card (#6788) Fix range of possible score, add inference . --- .../bert-multilingual-passage-reranking-msmarco/README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/model_cards/amberoad/bert-multilingual-passage-reranking-msmarco/README.md b/model_cards/amberoad/bert-multilingual-passage-reranking-msmarco/README.md index 1013acd15f..6539f3edcd 100644 --- a/model_cards/amberoad/bert-multilingual-passage-reranking-msmarco/README.md +++ b/model_cards/amberoad/bert-multilingual-passage-reranking-msmarco/README.md @@ -27,13 +27,13 @@ It can be used as an improvement for Elasticsearch Results and boosts the releva **Architecture:** On top of BERT there is a Densly Connected NN which takes the 768 Dimensional [CLS] Token as input and provides the output ([Arxiv](https://arxiv.org/abs/1901.04085)). -**Output:** Just a single value between between 0-1 +**Output:** Just a single value between between -10 and 10. Better matching query,passage pairs tend to have a higher a score. ## Intended uses & limitations Both query[1] and passage[2] have to fit in 512 Tokens. -As you normally want to rerank the first dozens of search results keep in mind the inference time. +As you normally want to rerank the first dozens of search results keep in mind the inference time of approximately 300 ms/query. #### How to use @@ -70,7 +70,7 @@ We see nearly similar performance than the English only Model in the English [Bi Fine-tuned Models | Dependency | Eval Set | Search Boost | Speed on GPU ----------------------------------------------------------------------------------- | ---------------------------------------------------------------------------- | ------------------------------------------------------------------ | ----------------------------------------------------- | ---------------------------------- -**`amberoad/Multilingual-uncased-MSMARCO`** (This Model) | PyTorch | bing queries | **+61%** (0.29 vs 0.18) | - +**`amberoad/Multilingual-uncased-MSMARCO`** (This Model) | PyTorch | bing queries | **+61%** (0.29 vs 0.18) | ~300 ms/query `nboost/pt-tinybert-msmarco` | PyTorch | bing queries | **+45%** (0.26 vs 0.18) | ~50ms/query `nboost/pt-bert-base-uncased-msmarco` | PyTorch | bing queries | **+62%** (0.29 vs 0.18) | ~300 ms/query `nboost/pt-bert-large-msmarco` | PyTorch | bing queries | **+77%** (0.32 vs 0.18) | -