SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity

Add tokenize_kwargs parameter definition in the FeatureExtractionPipeline (#22031)

Browse Source 
add tokenize_kwargs doc in the FeatureExtractionPipeline
This commit is contained in:
anruijian
2023-03-08 11:43:31 -05:00
committed by GitHub
parent a5392ee747
commit b427b263e2
1 changed files with 2 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
src/transformers/pipelines/feature_extraction.py
View File

@@ -53,6 +53,8 @@ class FeatureExtractionPipeline(Pipeline):
device (`int`, *optional*, defaults to -1): device (`int`, *optional*, defaults to -1):
Device ordinal for CPU/GPU supports. Setting this to -1 will leverage CPU, a positive will run the model on Device ordinal for CPU/GPU supports. Setting this to -1 will leverage CPU, a positive will run the model on
the associated CUDA device id. the associated CUDA device id.
tokenize_kwargs (`dict`, *optional*):
Additional dictionary of keyword arguments passed along to the tokenizer.
""" """
def _sanitize_parameters(self, truncation=None, tokenize_kwargs=None, return_tensors=None, **kwargs): def _sanitize_parameters(self, truncation=None, tokenize_kwargs=None, return_tensors=None, **kwargs):