Guide to fixed-length model perplexity evaluation (#5449)

Browse Source

* add first draft ppl guide

* upload imgs

* expand on strides

* ref typo

* rm superfluous past var

* add tokenization disclaimer

...

This commit is contained in:

Joe Davison

2020-07-07 16:04:15 -06:00

committed by GitHub

parent fde217c679

commit b4b33fdf25

5 changed files with 152 additions and 0 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download Patch File Download Diff File Expand all files Collapse all files

BIN

docs/source/imgs/ppl_chunked.gif Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 352 KiB

BIN

docs/source/imgs/ppl_full.gif Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 418 KiB

BIN

docs/source/imgs/ppl_sliding.gif Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 373 KiB