SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity

fix python 2.7 imports

Browse Source
This commit is contained in:
thomwolf
2019-02-11 10:35:36 +01:00
parent 8197eb9f10
commit 2071a9b86e
3 changed files with 15 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

13
README.md
View File

@@ -45,6 +45,12 @@ PyTorch pretrained bert can be installed by pip as follows:
pip install pytorch-pretrained-bert
```
If you want to use the tokenizer associated to the `OpenAI GPT` tokenizer, you will need to install `ftfy` (if you are using Python 2, version 4.4.3 is the last version working for you) and `SpaCy` :
```bash
pip install spacy ftfy==4.4.3
python -m spacy download en
```
### From source
Clone the repository and run:
@@ -52,6 +58,13 @@ Clone the repository and run:
pip install [--editable] .
```
Here also, if you want to use `OpenAIGPT` tokenizer, you will need to install `ftfy` (limit to version 4.4.3 if you are using Python 2) and `SpaCy` :
```bash
pip install spacy ftfy==4.4.3
python -m spacy download en
```
A series of tests is included in the [tests folder](https://github.com/huggingface/pytorch-pretrained-BERT/tree/master/tests) and can be run using `pytest` (install pytest if needed: `pip install pytest`).
You can run the tests with the command: