From 34561e61a5ce792f9e556217369792ec996a2fac Mon Sep 17 00:00:00 2001 From: thomwolf Date: Wed, 27 Mar 2019 12:00:04 +0100 Subject: [PATCH] update main readme also --- README.md | 13 +------------ 1 file changed, 1 insertion(+), 12 deletions(-) diff --git a/README.md b/README.md index afc2b6efda..85c990c122 100644 --- a/README.md +++ b/README.md @@ -1051,18 +1051,7 @@ You can download an [exemplary training corpus](https://ext-bert-sample.obs.eu-d Training one epoch on this corpus takes about 1:20h on 4 x NVIDIA Tesla P100 with `train_batch_size=200` and `max_seq_length=128`: -```shell -python run_lm_finetuning.py \ - --bert_model bert-base-uncased \ - --do_lower_case \ - --do_train \ - --train_file ../samples/sample_text.txt \ - --output_dir models \ - --num_train_epochs 5.0 \ - --learning_rate 3e-5 \ - --train_batch_size 32 \ - --max_seq_length 128 \ -``` +Thank to the work of @Rocketknight1 and @tholor there are now **several scripts** that can be used to fine-tune BERT using the pretraining objective (combination of masked-language modeling and next sentence prediction loss). These scripts are detailed in the [`README`](./examples/lm_finetuning/README.md) of the [`examples/lm_finetuning/`](./examples/lm_finetuning/) folder. ### OpenAI GPT, Transformer-XL and GPT-2: running the examples