@@ -254,7 +254,7 @@ For users, a rule of thumb is:
|
|||||||
## Pipeline chunk batching
|
## Pipeline chunk batching
|
||||||
|
|
||||||
`zero-shot-classification` and `question-answering` are slightly specific in the sense, that a single input might yield
|
`zero-shot-classification` and `question-answering` are slightly specific in the sense, that a single input might yield
|
||||||
mutliple forward pass of a model. Under normal circumstances, this would yield issues with `batch_size` argument.
|
multiple forward pass of a model. Under normal circumstances, this would yield issues with `batch_size` argument.
|
||||||
|
|
||||||
In order to circumvent this issue, both of these pipelines are a bit specific, they are `ChunkPipeline` instead of
|
In order to circumvent this issue, both of these pipelines are a bit specific, they are `ChunkPipeline` instead of
|
||||||
regular `Pipeline`. In short:
|
regular `Pipeline`. In short:
|
||||||
@@ -263,7 +263,7 @@ regular `Pipeline`. In short:
|
|||||||
```python
|
```python
|
||||||
preprocessed = pipe.preprocess(inputs)
|
preprocessed = pipe.preprocess(inputs)
|
||||||
model_outputs = pipe.forward(preprocessed)
|
model_outputs = pipe.forward(preprocessed)
|
||||||
outputs = pipe.postprocess(model_ouputs)
|
outputs = pipe.postprocess(model_outputs)
|
||||||
```
|
```
|
||||||
|
|
||||||
Now becomes:
|
Now becomes:
|
||||||
@@ -274,7 +274,7 @@ all_model_outputs = []
|
|||||||
for preprocessed in pipe.preprocess(inputs):
|
for preprocessed in pipe.preprocess(inputs):
|
||||||
model_outputs = pipe.forward(preprocessed)
|
model_outputs = pipe.forward(preprocessed)
|
||||||
all_model_outputs.append(model_outputs)
|
all_model_outputs.append(model_outputs)
|
||||||
outputs = pipe.postprocess(all_model_ouputs)
|
outputs = pipe.postprocess(all_model_outputs)
|
||||||
```
|
```
|
||||||
|
|
||||||
This should be very transparent to your code because the pipelines are used in
|
This should be very transparent to your code because the pipelines are used in
|
||||||
@@ -282,7 +282,7 @@ the same way.
|
|||||||
|
|
||||||
This is a simplified view, since the pipeline can handle automatically the batch to ! Meaning you don't have to care
|
This is a simplified view, since the pipeline can handle automatically the batch to ! Meaning you don't have to care
|
||||||
about how many forward passes you inputs are actually going to trigger, you can optimize the `batch_size`
|
about how many forward passes you inputs are actually going to trigger, you can optimize the `batch_size`
|
||||||
independantly of the inputs. The caveats from the previous section still apply.
|
independently of the inputs. The caveats from the previous section still apply.
|
||||||
|
|
||||||
## Pipeline custom code
|
## Pipeline custom code
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user