Add tokenizer kwargs to fill mask pipeline. (#26234)

Browse Source

* add tokenizer kwarg inputs

* Adding tokenizer_kwargs to _sanitize_parameters

* Add truncation=True example to tests

* Update test_pipelines_fill_mask.py

* Update test_pipelines_fill_mask.py

* make fix-copies and make style

* Update fill_mask.py

Replace single tick with double

* make fix-copies

* Style

---------

Co-authored-by: Lysandre <lysandre@huggingface.co>

...

This commit is contained in:

Nathan Cahill

2023-10-03 01:25:10 -07:00

committed by GitHub

parent df6a855e7b

commit b5ca8fcd20

4 changed files with 50 additions and 7 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download Patch File Download Diff File Expand all files Collapse all files

2

tests/models/markuplm/test_processor_markuplm.py

File diff suppressed because one or more lines are too long