Framework split (#16030)

* First files

* More files

* Last files

* Style
This commit is contained in:
Sylvain Gugger
2022-03-15 10:13:34 -04:00
committed by GitHub
parent 4a353cacb7
commit 4f4e5ddbcb
17 changed files with 465 additions and 132 deletions

View File

@@ -149,6 +149,9 @@ Finally, you want the tokenizer to return the actual tensors that are fed to the
Set the `return_tensors` parameter to either `pt` for PyTorch, or `tf` for TensorFlow:
<frameworkcontent>
<pt>
```py
>>> batch_sentences = [
... "But what about second breakfast?",
@@ -163,7 +166,10 @@ Set the `return_tensors` parameter to either `pt` for PyTorch, or `tf` for Tenso
[0, 0, 0, 0, 0, 0, 0, 0, 0]]),
'attention_mask': tensor([[1, 1, 1, 1, 1, 1, 1, 1, 1],
[1, 1, 1, 1, 1, 1, 1, 1, 0]])}
===PT-TF-SPLIT===
```
</pt>
<tf>
```py
>>> batch_sentences = [
... "But what about second breakfast?",
... "Don't think he knows about second breakfast, Pip.",
@@ -182,6 +188,8 @@ array([[0, 0, 0, 0, 0, 0, 0, 0, 0],
array([[1, 1, 1, 1, 1, 1, 1, 1, 1],
[1, 1, 1, 1, 1, 1, 1, 1, 0]], dtype=int32)>}
```
</tf>
</frameworkcontent>
## Audio