Sylvain Gugger e9c3031463 Fixes to make life easier with the nlp library (#6423) ...

* allow using tokenizer.pad as a collate_fn in pytorch

* allow using tokenizer.pad as a collate_fn in pytorch

* Add documentation and tests

* Make attention mask the right shape

* Better test

Co-authored-by: Thomas Wolf <thomwolf@users.noreply.github.com>

2020-08-12 08:00:56 -04:00

9.7 KiB

Raw Blame History

View Raw