@@ -254,7 +254,7 @@ For users, a rule of thumb is:
|
||||
## Pipeline chunk batching
|
||||
|
||||
`zero-shot-classification` and `question-answering` are slightly specific in the sense, that a single input might yield
|
||||
mutliple forward pass of a model. Under normal circumstances, this would yield issues with `batch_size` argument.
|
||||
multiple forward pass of a model. Under normal circumstances, this would yield issues with `batch_size` argument.
|
||||
|
||||
In order to circumvent this issue, both of these pipelines are a bit specific, they are `ChunkPipeline` instead of
|
||||
regular `Pipeline`. In short:
|
||||
@@ -263,7 +263,7 @@ regular `Pipeline`. In short:
|
||||
```python
|
||||
preprocessed = pipe.preprocess(inputs)
|
||||
model_outputs = pipe.forward(preprocessed)
|
||||
outputs = pipe.postprocess(model_ouputs)
|
||||
outputs = pipe.postprocess(model_outputs)
|
||||
```
|
||||
|
||||
Now becomes:
|
||||
@@ -274,7 +274,7 @@ all_model_outputs = []
|
||||
for preprocessed in pipe.preprocess(inputs):
|
||||
model_outputs = pipe.forward(preprocessed)
|
||||
all_model_outputs.append(model_outputs)
|
||||
outputs = pipe.postprocess(all_model_ouputs)
|
||||
outputs = pipe.postprocess(all_model_outputs)
|
||||
```
|
||||
|
||||
This should be very transparent to your code because the pipelines are used in
|
||||
@@ -282,7 +282,7 @@ the same way.
|
||||
|
||||
This is a simplified view, since the pipeline can handle automatically the batch to ! Meaning you don't have to care
|
||||
about how many forward passes you inputs are actually going to trigger, you can optimize the `batch_size`
|
||||
independantly of the inputs. The caveats from the previous section still apply.
|
||||
independently of the inputs. The caveats from the previous section still apply.
|
||||
|
||||
## Pipeline custom code
|
||||
|
||||
|
||||
Reference in New Issue
Block a user