From 93dccf527b81b807c3ec8c658a7bd7e9fca8d3f4 Mon Sep 17 00:00:00 2001 From: Lysandre Date: Thu, 30 Jan 2020 09:49:11 -0500 Subject: [PATCH] Pretrained models --- docs/source/pretrained_models.rst | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/docs/source/pretrained_models.rst b/docs/source/pretrained_models.rst index 39e942e3e4..e124e414c9 100644 --- a/docs/source/pretrained_models.rst +++ b/docs/source/pretrained_models.rst @@ -251,6 +251,22 @@ For a list that includes community-uploaded models, refer to `https://huggingfac | | ``xlm-roberta-large`` | | ~355M parameters with 24-layers, 1027-hidden-state, 4096 feed-forward hidden-state, 16-heads, | | | | | Trained on 2.5 TB of newly created clean CommonCrawl data in 100 languages | +-------------------+------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------+ +| FlauBERT | ``flaubert-small-cased`` | | 6-layer, 512-hidden, 8-heads, 54M parameters | +| | | | FlauBERT small architecture | +| | | (see `details `__) | +| +------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------+ +| | ``flaubert-base-uncased`` | | 12-layer, 768-hidden, 12-heads, 137M parameters | +| | | | FlauBERT base architecture with uncased vocabulary | +| | | (see `details `__) | +| +------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------+ +| | ``flaubert-base-cased`` | | 12-layer, 768-hidden, 12-heads, 138M parameters | +| | | | FlauBERT base architecture with cased vocabulary | +| | | (see `details `__) | +| +------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------+ +| | ``flaubert-large-cased`` | | 24-layer, 1024-hidden, 16-heads, 373M parameters | +| | | | FlauBERT large architecture | +| | | (see `details `__) | ++-------------------+------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------+ .. `__