Files
HuggingFace_transformer/tests/models
Thibault Douzon 4e441e529c fix LayoutLMv3TokenizerFast subword label after 'Ġ' token (#21695)
LayoutLMv3TokenizerFast produces empty 'Ġ' token with `offset_mapping = (0, 0)`.
Next token is wrongly assumed to also be beginning of word and isn't
correctly assigned `pad_token_label`.
Modify test with text that produce 'Ġ' token.
Remove copy check from LayoutLMv2TokenizerFast for `_batch_encode_plus`.

solves issue: #19978
2023-04-03 10:32:36 -04:00
..
2023-03-03 14:47:09 +01:00
2022-05-03 14:42:02 +02:00
2023-03-01 11:11:04 +01:00
2023-03-14 12:08:14 +03:00
2022-05-03 14:42:02 +02:00
2023-02-20 16:37:11 +03:00
2023-03-07 04:20:14 +01:00
2023-04-03 09:07:32 -04:00
2022-05-12 16:25:55 -04:00
2023-02-09 14:46:15 +00:00
2023-03-24 19:45:57 +00:00
2022-05-03 14:42:02 +02:00