Files
HuggingFace_transformer/tests
Joel Tang dbf7bfafa7 Fix idx2sym not loaded from pretrained vocab file in Transformer XL (#27589)
* Load idx2sym from pretrained vocab file in Transformer XL

When loading vocab file from a pretrained tokenizer for Transformer XL,
although the pickled vocabulary file contains a idx2sym key, it isn't
loaded, because it is discarded as the empty list already exists as
an attribute.

Solution is to explicitly take it into account, just like for sym2idx.

* ran make style
2023-11-20 07:56:18 +01:00
..
2023-10-04 15:13:37 +02:00
2023-11-13 14:20:54 +01:00