From 0eecaceac7a2cb3c067a435a7571a2ee0de619b9 Mon Sep 17 00:00:00 2001 From: Rodolfo De Nadai Date: Sun, 30 Aug 2020 08:02:46 -0300 Subject: [PATCH] BR_BERTo model card (#6793) --- model_cards/rdenadai/BR_BERTo/README.md | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-) diff --git a/model_cards/rdenadai/BR_BERTo/README.md b/model_cards/rdenadai/BR_BERTo/README.md index 59609b9fb7..594df42bf5 100644 --- a/model_cards/rdenadai/BR_BERTo/README.md +++ b/model_cards/rdenadai/BR_BERTo/README.md @@ -14,13 +14,17 @@ Portuguese (Brazil) model for text inference. ## Params -Trained on a corpus of 5_258_624 sentences, with 132_807_374 non unique tokens (992_418 unique tokens). +Trained on a corpus of 6_993_330 sentences. -- Vocab size: 220_000 -- RobertaForMaskedLM size : 32 -- Num train epochs: 2 -- Time to train: ~23hs (on GCP with a Nvidia T4) +- Vocab size: 150_000 +- RobertaForMaskedLM size : 512 +- Num train epochs: 3 +- Time to train: ~10days (on GCP with a Nvidia T4) I follow the great tutorial from HuggingFace team: [How to train a new language model from scratch using Transformers and Tokenizers](https://huggingface.co/blog/how-to-train) + +More infor here: + +[BR_BERTo](https://github.com/rdenadai/BR-BERTo)