SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity

[docstring] Fix bert generation tokenizer (#26820)

Browse Source 
* Remove BertGenerationTokenizer from objects to ignore

The file BertGenerationTokenizer is removed from
objects to ignore as a first step to fix docstring.

* Docstrings fix for BertGenerationTokenizer

Docstring fix is generated for BertGenerationTokenizer
by using check_docstrings.py.

* Fix docstring for BertGenerationTokenizer

Added sep_token type and docstring in BertGenerationTokenizer.
This commit is contained in:
przemL
2023-10-16 18:26:55 +02:00
committed by GitHub
parent 12cc123359
commit 5c6b83cb69
2 changed files with 6 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
utils/check_docstrings.py
View File

@@ -94,7 +94,6 @@ OBJECTS_TO_IGNORE = [
"BarthezTokenizerFast",
"BeitModel",
"BertConfig",
"BertGenerationTokenizer",
"BertJapaneseTokenizer",
"BertModel",
"BertTokenizerFast",