Files
HuggingFace_transformer/tests/models
Nilesh 4d4febb7aa Added test cases for rembert refering to albert and reformer test_tok… (#27637)
* Added test cases for rembert refering to albert and reformer test_tokenization

* removed CURL_CA_BUNDLE='

* Added flag test_sentencepiece_ignore_case and space_between_special_tokens to True

* Overrided test_added_tokens_serialization

* As slow->fast token failed due to the different initialization for [MASK]  for slow and fast, Therefore it required to make the initialization for [MASK] token uniform between fast and slow token

* Added few more test cases in test_encode_decode_round_trip and modefied the slow token (mask_token) to  have AddedToken instance with lstrip=True

* Added few test cases in test_encoder_decoder round trip and also modified slow tokenizer of rembert to have mask_token as AddedToken with lstrip = True

* Cleaned the code and added  fmt: skip to avoid line breaks after make style +  added comments to indicate from the copied test cases

* Corrected few comments

* Fixed quality issue

* Ran fix-copies

* Fixed few minor issues as (make fix-copies) broke few test cases while stripping the text

* Reverted the changes made by repo-consistancy

---------

Co-authored-by: Kokane <kokanen@apac.corpdir.net>
2023-12-04 13:36:57 +01:00
..
2023-11-13 14:20:54 +01:00
2023-11-28 08:38:32 +00:00
2023-11-28 17:40:01 +01:00
2023-10-24 16:49:26 +02:00
2023-08-02 20:22:36 +02:00
2023-08-02 20:22:36 +02:00
2023-09-05 11:19:56 +02:00
2023-08-02 20:22:36 +02:00
2023-08-02 20:22:36 +02:00
2023-11-16 17:43:19 +01:00
2023-06-16 15:40:49 +01:00
2023-08-02 20:22:36 +02:00
2023-08-02 20:22:36 +02:00
2023-08-02 20:22:36 +02:00
2023-08-02 20:22:36 +02:00
2023-09-18 20:28:36 +02:00
2023-05-24 13:52:52 +01:00
2023-11-08 13:26:02 +00:00
2023-11-16 17:43:19 +01:00
2023-08-02 20:22:36 +02:00
2023-09-26 07:06:04 +02:00
2023-08-02 20:22:36 +02:00
2023-08-02 20:22:36 +02:00
2023-08-02 20:22:36 +02:00
2023-08-02 20:22:36 +02:00
2023-06-16 15:40:49 +01:00
2023-08-02 20:22:36 +02:00
2023-08-02 20:22:36 +02:00
2023-11-30 20:24:43 +01:00
2023-09-05 10:12:25 +02:00
2023-09-05 10:12:25 +02:00
2023-08-02 20:22:36 +02:00
2023-08-02 20:22:36 +02:00
2023-08-02 20:22:36 +02:00
2023-12-04 09:40:42 +01:00
2023-08-16 17:45:02 +01:00
2023-08-02 20:22:36 +02:00
2023-08-02 20:22:36 +02:00
2023-08-02 20:22:36 +02:00
2023-08-02 20:22:36 +02:00
2022-05-03 14:42:02 +02:00