Kevin Canwen Xu fb41f9f50c Add a special tokenizer for CPM model (#11068) ...

* Add a special tokenizer for CPM model

* make style

* fix

* Add docs

* styles

* cpm doc

* fix ci

* fix the overview

* add test

* make style

* typo

* Custom tokenizer flag

* Add REAMDE.md

Co-authored-by: Lysandre <lysandre.debut@reseau.eseo.fr>

2021-04-10 02:07:47 +08:00

1.7 KiB

Raw Blame History

View Raw