🚨🚨🚨 [Pix2Struct] Attempts to fix training issues 🚨🚨🚨 (#23004)
* multiple fixes - add `add_special_tokens` to `True` by default - remove label smoothing and labels masking * fix test
This commit is contained in:
@@ -108,7 +108,7 @@ class Pix2StructProcessorTest(unittest.TestCase):
|
||||
|
||||
encoded_processor = processor(text=input_str)
|
||||
|
||||
encoded_tok = tokenizer(input_str, return_token_type_ids=False, add_special_tokens=False)
|
||||
encoded_tok = tokenizer(input_str, return_token_type_ids=False, add_special_tokens=True)
|
||||
|
||||
for key in encoded_tok.keys():
|
||||
self.assertListEqual(encoded_tok[key], encoded_processor[key])
|
||||
|
||||
Reference in New Issue
Block a user