From e239a4a20fbb901e60ffcafc06bfefcbb67eaa65 Mon Sep 17 00:00:00 2001 From: thomwolf Date: Tue, 20 Aug 2019 11:02:00 +0200 Subject: [PATCH] close #984 --- docs/source/pretrained_models.rst | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/source/pretrained_models.rst b/docs/source/pretrained_models.rst index 987882d12e..6a14e3dcd1 100644 --- a/docs/source/pretrained_models.rst +++ b/docs/source/pretrained_models.rst @@ -72,16 +72,16 @@ Here is the full list of the currently provided pretrained models together with | | ``xlnet-large-cased`` | | 24-layer, 1024-hidden, 16-heads, 340M parameters. | | | | | XLNet Large English model | +-------------------+------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------+ -| XLM | ``xlm-mlm-en-2048`` | | 12-layer, 1024-hidden, 8-heads | +| XLM | ``xlm-mlm-en-2048`` | | 12-layer, 2048-hidden, 16-heads | | | | | XLM English model | | +------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------+ -| | ``xlm-mlm-ende-1024`` | | 12-layer, 1024-hidden, 8-heads | +| | ``xlm-mlm-ende-1024`` | | 6-layer, 1024-hidden, 8-heads | | | | | XLM English-German Multi-language model | | +------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------+ -| | ``xlm-mlm-enfr-1024`` | | 12-layer, 1024-hidden, 8-heads | +| | ``xlm-mlm-enfr-1024`` | | 6-layer, 1024-hidden, 8-heads | | | | | XLM English-French Multi-language model | | +------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------+ -| | ``xlm-mlm-enro-1024`` | | 12-layer, 1024-hidden, 8-heads | +| | ``xlm-mlm-enro-1024`` | | 6-layer, 1024-hidden, 8-heads | | | | | XLM English-Romanian Multi-language model | | +------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------+ | | ``xlm-mlm-xnli15-1024`` | | 12-layer, 1024-hidden, 8-heads | @@ -93,7 +93,7 @@ Here is the full list of the currently provided pretrained models together with | | ``xlm-clm-enfr-1024`` | | 12-layer, 1024-hidden, 8-heads | | | | | XLM English model trained with CLM (Causal Language Modeling) | | +------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------+ -| | ``xlm-clm-ende-1024`` | | 12-layer, 1024-hidden, 8-heads | +| | ``xlm-clm-ende-1024`` | | 6-layer, 1024-hidden, 8-heads | | | | | XLM English-German Multi-language model trained with CLM (Causal Language Modeling) | +-------------------+------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------+ | RoBERTa | ``roberta-base`` | | 12-layer, 768-hidden, 12-heads, 125M parameters |