SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity

Add hugs (#5225)

Browse Source
This commit is contained in:
Sylvain Gugger
2020-06-24 07:56:14 -04:00
committed by GitHub
parent 5e85b324ec
commit 7c41057d50
9 changed files with 39 additions and 40 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
docs/source/index.rst
View File

@@ -42,7 +42,7 @@ The documentation is organized in five parts:
- **GET STARTED** contains a quick tour, the installation instructions and some useful information about our philosophy
and a glossary.
- **USING TRANSFORMERS** contains general tutorials on how to use the library.
- **USING 🤗 TRANSFORMERS** contains general tutorials on how to use the library.
- **ADVANCED GUIDES** contains more advanced guides that are more specific to a given script or part of the library.
- **RESEARCH** focuses on tutorials that have less to do with how to use the library but more about general resarch in
transformers model
@@ -135,7 +135,7 @@ conversion utilities for the following models:
.. toctree::
:maxdepth: 2
:caption: Using Transformers
:caption: Using 🤗 Transformers
task_summary
model_summary

16
docs/source/migration.md
View File

@@ -1,8 +1,8 @@
# Migrating from previous packages
## Migrating from pytorch-transformers to transformers
## Migrating from pytorch-transformers to 🤗 Transformers
Here is a quick summary of what you should take care of when migrating from `pytorch-transformers` to `transformers`.
Here is a quick summary of what you should take care of when migrating from `pytorch-transformers` to 🤗 Transformers.
### Positional order of some models' keywords inputs (`attention_mask`, `token_type_ids`...) changed
@@ -14,17 +14,17 @@ If you used to call the models with positional inputs for keyword arguments, e.g
## Migrating from pytorch-pretrained-bert
Here is a quick summary of what you should take care of when migrating from `pytorch-pretrained-bert` to `transformers`
Here is a quick summary of what you should take care of when migrating from `pytorch-pretrained-bert` to 🤗 Transformers
### Models always output `tuples`
The main breaking change when migrating from `pytorch-pretrained-bert` to `transformers` is that the models forward method always outputs a `tuple` with various elements depending on the model and the configuration parameters.
The main breaking change when migrating from `pytorch-pretrained-bert` to 🤗 Transformers is that the models forward method always outputs a `tuple` with various elements depending on the model and the configuration parameters.
The exact content of the tuples for each model are detailled in the models' docstrings and the [documentation](https://huggingface.co/transformers/).
In pretty much every case, you will be fine by taking the first element of the output as the output you previously used in `pytorch-pretrained-bert`.
Here is a `pytorch-pretrained-bert` to `transformers` conversion example for a `BertForSequenceClassification` classification model:
Here is a `pytorch-pretrained-bert` to 🤗 Transformers conversion example for a `BertForSequenceClassification` classification model:
```python
# Let's load our model
@@ -33,11 +33,11 @@ model = BertForSequenceClassification.from_pretrained('bert-base-uncased')
# If you used to have this line in pytorch-pretrained-bert:
loss = model(input_ids, labels=labels)
# Now just use this line in transformers to extract the loss from the output tuple:
# Now just use this line in 🤗 Transformers to extract the loss from the output tuple:
outputs = model(input_ids, labels=labels)
loss = outputs[0]
# In transformers you can also have access to the logits:
# In 🤗 Transformers you can also have access to the logits:
loss, logits = outputs[:2]
# And even the attention weights if you configure the model to output them (and other outputs too, see the docstrings and documentation)
@@ -109,7 +109,7 @@ for batch in train_data:
loss.backward()
optimizer.step()
### In Transformers, optimizer and schedules are splitted and instantiated like this:
### In 🤗 Transformers, optimizer and schedules are splitted and instantiated like this:
optimizer = AdamW(model.parameters(), lr=lr, correct_bias=False) # To reproduce BertAdam specific behavior set correct_bias=False
scheduler = get_linear_schedule_with_warmup(optimizer, num_warmup_steps=num_warmup_steps, num_training_steps=num_training_steps) # PyTorch scheduler
### and used like this:

2
docs/source/model_summary.rst
View File

@@ -1,7 +1,7 @@
Summary of the models
================================================
This is a summary of the models available in the transformers library. It assumes you’re familiar with the original
This is a summary of the models available in 🤗 Transformers. It assumes you’re familiar with the original
`transformer model <https://arxiv.org/abs/1706.03762>`_. For a gentle introduction check the `annotated transformer
<http://nlp.seas.harvard.edu/2018/04/03/attention.html>`_. Here we focus on the high-level differences between the
models. You can check them more in detail in their respective documentation. Also checkout the

2
docs/source/philosophy.rst
View File

@@ -1,7 +1,7 @@
Philosophy
==========
Transformers is an opinionated library built for:
🤗 Transformers is an opinionated library built for:
- NLP researchers and educators seeking to use/study/extend large-scale transformers models
- hands-on practitioners who want to fine-tune those models and/or serve them in production

27
docs/source/quicktour.rst
View File

@@ -44,27 +44,27 @@ make them readable. For instance
::
classifier('We are very happy to show you the Transformers library.')
classifier('We are very happy to show you the 🤗 Transformers library.')
will return something like this:
::
[{'label': 'POSITIVE', 'score': 0.999799370765686}]
[{'label': 'POSITIVE', 'score': 0.9997795224189758}]
That's encouraging! You can use it on a list of sentences, which will be preprocessed then fed to the model as a
`batch`:
::
classifier(["We are very happy to show you the Transformers library.",
classifier(["We are very happy to show you the 🤗 Transformers library.",
"We hope you don't hate it."])
returning a list of dictionaries like this one:
::
[{'label': 'POSITIVE', 'score': 0.999799370765686},
[{'label': 'POSITIVE', 'score': 0.9997795224189758},
{'label': 'NEGATIVE', 'score': 0.5308589935302734}]
You can see the second sentence has been classified as negative (it needs to be positive or negative) but its score is
@@ -163,7 +163,7 @@ To apply these steps on a given text, we can just feed it to our tokenizer:
::
input = tokenizer("We are very happy to show you the Transformers library.")
input = tokenizer("We are very happy to show you the 🤗 Transformers library.")
print(input)
This returns a dictionary string to list of ints. It contains the `ids of the tokens <glossary.html#input-ids>`__,
@@ -172,9 +172,8 @@ as mentioned before, but also additional arguments that will be useful to the mo
::
{'input_ids': [101, 2057, 2024, 2200, 3407, 2000, 2265, 2017, 1996, 19081, 3075, 1012, 102],
'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]}
{'input_ids': [101, 2057, 2024, 2200, 3407, 2000, 2265, 2017, 1996, 100, 19081, 3075, 1012, 102],
'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]}
You can pass a list of sentences directly to your tokenizer. If your goal is to send them through your model as a
batch, you probably want to pad them all to the same length, truncate them to the maximum length the model can accept
@@ -184,13 +183,13 @@ and get tensors back. You can specify all of that to the tokenizer:
## PYTORCH CODE
batch = tokenizer(
["We are very happy to show you the Transformers library.",
["We are very happy to show you the 🤗 Transformers library.",
"We hope you don't hate it."],
padding=True, truncation=True, return_tensors="pt")
print(batch)
## TENSORFLOW CODE
batch = tokenizer(
["We are very happy to show you the Transformers library.",
["We are very happy to show you the 🤗 Transformers library.",
"We hope you don't hate it."],
padding=True, truncation=True, return_tensors="tf")
print(batch)
@@ -200,10 +199,10 @@ padding token the model was pretrained with. The attention mask is also adapted
::
{'input_ids': tensor([[ 101, 2057, 2024, 2200, 3407, 2000, 2265, 2017, 1996, 19081, 3075, 1012, 102],
[ 101, 2057, 3246, 2017, 2123, 1005, 1056, 5223, 2009, 1012, 102, 0, 0]]),
'attention_mask': tensor([[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1],
[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0]])}
{'input_ids': tensor([[ 101, 2057, 2024, 2200, 3407, 2000, 2265, 2017, 1996, 100, 19081, 3075, 1012, 102],
[ 101, 2057, 3246, 2017, 2123, 1005, 1056, 5223, 2009, 1012, 102, 0, 0, 0]]),
'attention_mask': tensor([[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1],
[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0]])}
You can learn more about tokenizers on their :doc:`doc page <main_classes/tokenizer>` (tutorial coming soon).

18
docs/source/task_summary.rst
View File

@@ -217,9 +217,9 @@ Here is an example of question answering using a model and a tokenizer. The proc
"""
questions = [
"How many pretrained models are available in Transformers?",
"What does Transformers provide?",
"Transformers provides interoperability between which frameworks?",
"How many pretrained models are available in 🤗 Transformers?",
"What does 🤗 Transformers provide?",
"🤗 Transformers provides interoperability between which frameworks?",
]
for question in questions:
@@ -253,9 +253,9 @@ Here is an example of question answering using a model and a tokenizer. The proc
"""
questions = [
"How many pretrained models are available in Transformers?",
"What does Transformers provide?",
"Transformers provides interoperability between which frameworks?",
"How many pretrained models are available in 🤗 Transformers?",
"What does 🤗 Transformers provide?",
"🤗 Transformers provides interoperability between which frameworks?",
]
for question in questions:
@@ -280,13 +280,13 @@ This outputs the questions followed by the predicted answers:
::
Question: How many pretrained models are available in Transformers?
Question: How many pretrained models are available in 🤗 Transformers?
Answer: over 32 +
Question: What does Transformers provide?
Question: What does 🤗 Transformers provide?
Answer: general - purpose architectures
Question: Transformers provides interoperability between which frameworks?
Question: 🤗 Transformers provides interoperability between which frameworks?
Answer: tensorflow 2 . 0 and pytorch

2
docs/source/torchscript.rst
View File

@@ -12,7 +12,7 @@ According to Pytorch's documentation: "TorchScript is a way to create serializab
Pytorch's two modules `JIT and TRACE <https://pytorch.org/docs/stable/jit.html>`_ allow the developer to export
their model to be re-used in other programs, such as efficiency-oriented C++ programs.
We have provided an interface that allows the export of `transformers` models to TorchScript so that they can
We have provided an interface that allows the export of 🤗 Transformers models to TorchScript so that they can
be reused in a different environment than a Pytorch-based python program. Here we explain how to use our models so that
they can be exported, and what to be mindful of when using these models with TorchScript.