Quentin Lhoest 9a8c168f56 Sort unique_no_split_tokens to make it deterministic (#6461) ...

* change unique_no_split_tokens's type to set

* use sorted list instead of set

* style

2020-08-14 10:36:58 +02:00

..

transformers

Sort unique_no_split_tokens to make it deterministic (#6461)

2020-08-14 10:36:58 +02:00