SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity

update main readme also

Browse Source
This commit is contained in:
thomwolf
2019-03-27 12:00:04 +01:00
parent 361aff6de5
commit 34561e61a5
1 changed files with 1 additions and 12 deletions
Download Patch File Download Diff File Expand all files Collapse all files

13
README.md
View File

@@ -1051,18 +1051,7 @@ You can download an [exemplary training corpus](https://ext-bert-sample.obs.eu-d
Training one epoch on this corpus takes about 1:20h on 4 x NVIDIA Tesla P100 with `train_batch_size=200` and `max_seq_length=128`: Training one epoch on this corpus takes about 1:20h on 4 x NVIDIA Tesla P100 with `train_batch_size=200` and `max_seq_length=128`:
```shell Thank to the work of @Rocketknight1 and @tholor there are now **several scripts** that can be used to fine-tune BERT using the pretraining objective (combination of masked-language modeling and next sentence prediction loss). These scripts are detailed in the [`README`](./examples/lm_finetuning/README.md) of the [`examples/lm_finetuning/`](./examples/lm_finetuning/) folder.
python run_lm_finetuning.py \
--bert_model bert-base-uncased \
--do_lower_case \
--do_train \
--train_file ../samples/sample_text.txt \
--output_dir models \
--num_train_epochs 5.0 \
--learning_rate 3e-5 \
--train_batch_size 32 \
--max_seq_length 128 \
```
### OpenAI GPT, Transformer-XL and GPT-2: running the examples ### OpenAI GPT, Transformer-XL and GPT-2: running the examples