diff --git a/docs/source/model_summary.rst b/docs/source/model_summary.rst index 4a675c797a..9ddf38f7b6 100644 --- a/docs/source/model_summary.rst +++ b/docs/source/model_summary.rst @@ -14,6 +14,7 @@ Each one of the models in the library falls into one of the following categories * :ref:`autoencoding-models` * :ref:`seq-to-seq-models` * :ref:`multimodal-models` + * :ref:`retrieval-based-models` Autoregressive models are pretrained on the classic language modeling task: guess the next token having read all the previous ones. They correspond to the decoder of the original transformer model, and a mask is used on top of the full @@ -605,6 +606,40 @@ The pretrained model only works for classification. More information in this :doc:`model documentation `. TODO: write this page +.. _retrieval-based-models: + +Retrieval-based models +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Some models use documents retrieval during (pre)training and inference for open-domain question answering, for example. + + +DPR +---------------------------------------------- + +.. raw:: html + + + Models + + + Doc + + +`Dense Passage Retrieval for Open-Domain Question Answering `_, +Vladimir Karpukhin et al. + +Dense Passage Retrieval (DPR) - is a set of tools and models for state-of-the-art open-domain question-answering research. + + +DPR consists in three models: + + * Question encoder: encode questions as vectors + * Context encoder: encode contexts as vectors + * Reader: extract the answer of the questions inside retrieved contexts, along with a relevance score (high if the inferred span actually answers the question). + +DPR's pipeline (not implemented yet) uses a retrieval step to find the top k contexts given a certain question, and then it calls the reader with the question and the retrieved documents to get the answer. + More technical aspects ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^