SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity

Create perplexity.rst (#13004)

Browse Source 
Updating the import for load_dataset
This commit is contained in:
Sasha Luccioni
2021-08-05 02:56:13 -04:00
committed by GitHub
parent 83e5a10603
commit 8aa01d2a6d
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
docs/source/perplexity.rst
View File

@@ -96,7 +96,7 @@ dataset in memory.
.. code-block:: python .. code-block:: python
from nlp import load_dataset from datasets import load_dataset
test = load_dataset('wikitext', 'wikitext-2-raw-v1', split='test') test = load_dataset('wikitext', 'wikitext-2-raw-v1', split='test')
encodings = tokenizer('\n\n'.join(test['text']), return_tensors='pt') encodings = tokenizer('\n\n'.join(test['text']), return_tensors='pt')