Nilesh
4d4febb7aa
Added test cases for rembert refering to albert and reformer test_tok… ( #27637 )
...
* Added test cases for rembert refering to albert and reformer test_tokenization
* removed CURL_CA_BUNDLE='
* Added flag test_sentencepiece_ignore_case and space_between_special_tokens to True
* Overrided test_added_tokens_serialization
* As slow->fast token failed due to the different initialization for [MASK] for slow and fast, Therefore it required to make the initialization for [MASK] token uniform between fast and slow token
* Added few more test cases in test_encode_decode_round_trip and modefied the slow token (mask_token) to have AddedToken instance with lstrip=True
* Added few test cases in test_encoder_decoder round trip and also modified slow tokenizer of rembert to have mask_token as AddedToken with lstrip = True
* Cleaned the code and added fmt: skip to avoid line breaks after make style + added comments to indicate from the copied test cases
* Corrected few comments
* Fixed quality issue
* Ran fix-copies
* Fixed few minor issues as (make fix-copies) broke few test cases while stripping the text
* Reverted the changes made by repo-consistancy
---------
Co-authored-by: Kokane <kokanen@apac.corpdir.net >
2023-12-04 13:36:57 +01:00
..
2023-11-16 17:43:19 +01:00
2023-10-30 19:53:46 +01:00
2023-11-16 17:43:19 +01:00
2023-11-08 07:39:37 +00:00
2023-11-13 14:20:54 +01:00
2023-10-30 19:53:46 +01:00
2023-11-16 17:43:19 +01:00
2023-10-31 18:12:14 +01:00
2023-11-16 17:43:19 +01:00
2023-02-06 18:10:56 -05:00
2023-11-28 08:38:32 +00:00
2023-10-31 18:12:14 +01:00
2023-11-16 17:43:19 +01:00
2023-11-16 17:43:19 +01:00
2023-02-06 18:10:56 -05:00
2023-11-16 17:43:19 +01:00
2023-10-31 18:12:14 +01:00
2023-10-11 14:12:09 +02:00
2023-11-27 15:48:17 +01:00
2023-10-31 18:12:14 +01:00
2023-10-31 18:12:14 +01:00
2023-10-31 18:12:14 +01:00
2023-10-31 18:12:14 +01:00
2023-11-16 17:43:19 +01:00
2023-08-11 11:30:18 +01:00
2023-09-25 18:08:12 +02:00
2023-11-16 17:43:19 +01:00
2023-11-16 17:43:19 +01:00
2023-11-16 17:43:19 +01:00
2023-11-16 17:43:19 +01:00
2023-10-30 19:53:46 +01:00
2023-11-16 17:43:19 +01:00
2023-11-16 17:43:19 +01:00
2023-11-28 17:40:01 +01:00
2023-11-16 17:43:19 +01:00
2023-10-24 16:49:26 +02:00
2023-08-11 11:30:18 +01:00
2023-10-31 18:12:14 +01:00
2023-11-27 15:48:17 +01:00
2023-11-27 15:48:17 +01:00
2023-03-22 20:02:24 +01:00
2023-08-02 20:22:36 +02:00
2023-10-31 18:12:14 +01:00
2023-11-27 15:48:17 +01:00
2023-11-27 15:48:17 +01:00
2023-08-02 20:22:36 +02:00
2023-11-16 17:43:19 +01:00
2023-02-28 19:40:57 +01:00
2023-10-31 18:12:14 +01:00
2023-11-27 15:48:17 +01:00
2023-08-11 11:30:18 +01:00
2023-09-05 11:19:56 +02:00
2023-11-27 15:48:17 +01:00
2023-11-27 15:48:17 +01:00
2023-11-09 13:32:54 +01:00
2023-06-29 10:17:36 +01:00
2023-11-27 15:48:17 +01:00
2023-08-02 20:22:36 +02:00
2023-11-27 15:48:17 +01:00
2023-11-27 15:48:17 +01:00
2023-11-27 15:48:17 +01:00
2023-08-02 20:22:36 +02:00
2023-10-04 17:09:48 +02:00
2023-11-16 17:43:19 +01:00
2023-10-31 18:12:14 +01:00
2023-11-16 17:43:19 +01:00
2023-11-16 17:43:19 +01:00
2023-11-24 09:43:20 +01:00
2023-10-31 18:12:14 +01:00
2023-11-16 17:43:19 +01:00
2023-11-16 17:43:19 +01:00
2023-11-27 15:48:17 +01:00
2023-10-31 18:12:14 +01:00
2023-06-16 15:40:49 +01:00
2023-11-23 21:00:39 +01:00
2023-10-30 19:53:46 +01:00
2023-11-27 15:48:17 +01:00
2023-11-16 17:43:19 +01:00
2023-08-02 20:22:36 +02:00
2023-11-16 17:43:19 +01:00
2023-08-02 20:22:36 +02:00
2023-11-16 17:43:19 +01:00
2023-11-16 17:43:19 +01:00
2023-11-16 17:43:19 +01:00
2023-11-16 17:43:19 +01:00
2023-07-24 10:08:47 -04:00
2023-10-30 19:53:46 +01:00
2023-10-18 16:30:53 +02:00
2023-10-05 10:27:05 +02:00
2023-11-08 20:58:36 +01:00
2023-11-21 13:26:01 +01:00
2023-10-30 19:53:46 +01:00
2023-10-30 19:53:46 +01:00
2023-11-16 17:43:19 +01:00
2023-10-31 18:12:14 +01:00
2023-11-23 21:00:39 +01:00
2023-10-30 19:53:46 +01:00
2023-11-16 17:43:19 +01:00
2023-11-16 17:43:19 +01:00
2023-11-16 17:43:19 +01:00
2023-10-31 18:12:14 +01:00
2023-11-27 15:48:17 +01:00
2023-10-30 19:53:46 +01:00
2023-11-21 23:20:44 +09:00
2023-10-11 14:12:09 +02:00
2023-08-02 20:22:36 +02:00
2023-10-30 19:53:46 +01:00
2023-11-16 17:43:19 +01:00
2023-11-16 17:43:19 +01:00
2023-11-16 17:43:19 +01:00
2023-11-16 17:43:19 +01:00
2023-11-27 15:48:17 +01:00
2023-11-27 15:48:17 +01:00
2023-10-31 18:12:14 +01:00
2023-11-16 17:43:19 +01:00
2023-10-31 18:12:14 +01:00
2023-08-02 20:22:36 +02:00
2022-05-03 14:42:02 +02:00
2023-11-27 15:48:17 +01:00
2023-11-27 10:18:41 +01:00
2023-09-18 20:28:36 +02:00
2023-11-14 10:34:03 +01:00
2023-11-27 15:48:17 +01:00
2023-11-27 15:48:17 +01:00
2023-11-27 15:48:17 +01:00
2023-11-27 15:48:17 +01:00
2023-11-13 15:18:19 +01:00
2023-08-21 11:11:21 +02:00
2023-10-30 19:53:46 +01:00
2023-05-24 13:52:52 +01:00
2023-11-08 13:26:02 +00:00
2023-11-16 17:43:19 +01:00
2023-11-27 15:48:17 +01:00
2023-08-02 20:22:36 +02:00
2023-11-16 17:43:19 +01:00
2023-11-27 14:23:54 +00:00
2023-09-26 07:06:04 +02:00
2023-08-02 20:22:36 +02:00
2023-11-16 17:43:19 +01:00
2023-08-02 20:22:36 +02:00
2023-10-31 18:12:14 +01:00
2023-12-04 09:45:22 +01:00
2023-11-16 17:43:19 +01:00
2023-11-29 13:36:38 +01:00
2023-11-16 17:43:19 +01:00
2023-10-31 18:12:14 +01:00
2023-11-16 17:43:19 +01:00
2023-10-31 18:12:14 +01:00
2023-11-28 17:21:21 +01:00
2023-02-06 18:10:56 -05:00
2023-10-30 19:53:46 +01:00
2023-10-31 18:12:14 +01:00
2023-11-27 15:48:17 +01:00
2023-09-25 18:08:12 +02:00
2023-08-02 20:22:36 +02:00
2023-11-27 15:48:17 +01:00
2023-08-02 20:22:36 +02:00
2023-06-16 15:40:49 +01:00
2023-08-02 20:22:36 +02:00
2023-11-16 17:43:19 +01:00
2023-11-27 15:48:17 +01:00
2023-12-04 13:36:57 +01:00
2023-11-27 15:48:17 +01:00
2023-10-11 14:12:09 +02:00
2023-11-16 17:43:19 +01:00
2023-10-11 14:12:09 +02:00
2023-10-30 19:53:46 +01:00
2023-08-02 20:22:36 +02:00
2023-11-27 15:48:17 +01:00
2023-11-30 20:24:43 +01:00
2023-11-30 20:24:43 +01:00
2023-11-27 15:48:17 +01:00
2023-09-05 10:12:25 +02:00
2023-09-05 10:12:25 +02:00
2023-02-06 18:10:56 -05:00
2023-11-16 17:43:19 +01:00
2023-11-14 10:34:03 +01:00
2023-11-16 17:43:19 +01:00
2023-08-02 20:22:36 +02:00
2023-08-02 20:22:36 +02:00
2023-11-27 15:48:17 +01:00
2023-11-27 15:48:17 +01:00
2023-11-27 15:48:17 +01:00
2023-11-27 15:48:17 +01:00
2023-11-27 14:23:54 +00:00
2023-11-27 14:23:54 +00:00
2023-07-25 07:56:04 -04:00
2023-11-16 17:43:19 +01:00
2023-10-30 19:53:46 +01:00
2023-11-27 15:48:17 +01:00
2023-09-18 23:47:54 +02:00
2023-08-02 20:22:36 +02:00
2023-11-09 18:35:42 +00:00
2023-12-04 09:40:42 +01:00
2023-10-30 19:53:46 +01:00
2023-10-03 10:52:34 +02:00
2023-10-03 10:52:34 +02:00
2023-11-22 17:21:36 +01:00
2023-11-27 15:48:17 +01:00
2023-11-27 15:48:17 +01:00
2023-10-30 19:53:46 +01:00
2023-11-16 17:43:19 +01:00
2023-11-16 17:43:19 +01:00
2023-10-30 19:53:46 +01:00
2023-11-27 15:48:17 +01:00
2023-11-27 15:48:17 +01:00
2023-11-27 15:48:17 +01:00
2023-11-27 15:48:17 +01:00
2023-11-27 15:48:17 +01:00
2023-11-27 15:48:17 +01:00
2023-10-03 19:43:42 +02:00
2023-08-16 17:45:02 +01:00
2023-11-16 17:43:19 +01:00
2023-10-31 18:12:14 +01:00
2022-05-03 14:42:02 +02:00
2023-10-30 15:27:15 +01:00
2023-10-03 10:52:34 +02:00
2023-11-23 21:00:39 +01:00
2023-10-30 19:53:46 +01:00
2023-11-16 17:43:19 +01:00
2023-08-02 20:22:36 +02:00
2023-11-16 17:43:19 +01:00
2023-11-16 17:43:19 +01:00
2023-08-02 20:22:36 +02:00
2023-11-16 17:43:19 +01:00
2023-08-02 20:22:36 +02:00
2023-11-27 15:48:17 +01:00
2023-08-02 20:22:36 +02:00
2022-05-03 14:42:02 +02:00