From 0ccbfd286876d6bac199cada5ae74246c4ed8a3e Mon Sep 17 00:00:00 2001 From: Patrick von Platen Date: Sat, 2 May 2020 10:31:00 +0200 Subject: [PATCH] Update Reformer ReadME --- .../google/reformer-crime-and-punishment/README.md | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-) diff --git a/model_cards/google/reformer-crime-and-punishment/README.md b/model_cards/google/reformer-crime-and-punishment/README.md index 0886727834..03d54279a5 100644 --- a/model_cards/google/reformer-crime-and-punishment/README.md +++ b/model_cards/google/reformer-crime-and-punishment/README.md @@ -1,10 +1,14 @@ ## Reformer Model trained on "Crime and Punishment" -Crime and Punishment text was taken from `gs://trax-ml/reformer/crime-and-punishment-2554.txt`. -Model was trained in flax using colab notebook proposed by authors: https://colab.research.google.com/github/google/trax/blob/master/trax/models/reformer/text_generation.ipynb -Weights were converted to Hugging Face PyTorch `ReformerModelWithLMHead`. -The model is a character level language model. Language from in `crime-and-punishment` style -can be generated as follows: +Crime and Punishment is a novel written by Fyodor Dostoevsky and was translated into English. + +Crime and Punishment training data was taken from `gs://trax-ml/reformer/crime-and-punishment-2554.txt` and contains +roughly 0.5M tokens. + +The ReformerLM model was trained in flax using colab notebook proposed by authors: https://colab.research.google.com/github/google/trax/blob/master/trax/models/reformer/text_generation.ipynb . +and the weights were converted to Hugging Face's PyTorch ReformerLM model `ReformerModelWithLMHead`. + +The model is a language model that operates on small sub-word units. Text can be generated as follows: ```python model = ReformerModelWithLMHead.from_pretrained("patrickvonplaten/reformer-crime-and-punish")