SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity

[docs] last hidden state vs hidden_states[-1] (#26142)

Browse Source 
* last hidden state clarification

* feedback addressed
This commit is contained in:
Maria Khalusova
2023-09-13 14:35:42 -04:00
committed by GitHub
parent e52f1cb669
commit 9709ab116c
1 changed files with 7 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

7
docs/source/en/main_classes/output.md
View File

@@ -40,6 +40,13 @@ an optional `attentions` attribute. Here we have the `loss` since we passed alon
`hidden_states` and `attentions` because we didn't pass `output_hidden_states=True` or `hidden_states` and `attentions` because we didn't pass `output_hidden_states=True` or
`output_attentions=True`. `output_attentions=True`.
<Tip>
When passing `output_hidden_states=True` you may expect the `outputs.hidden_states[-1]` to match `outputs.last_hidden_states` exactly.
However, this is not always the case. Some models apply normalization or subsequent process to the last hidden state when it's returned.
</Tip>
You can access each attribute as you would usually do, and if that attribute has not been returned by the model, you You can access each attribute as you would usually do, and if that attribute has not been returned by the model, you
will get `None`. Here for instance `outputs.loss` is the loss computed by the model, and `outputs.attentions` is will get `None`. Here for instance `outputs.loss` is the loss computed by the model, and `outputs.attentions` is
`None`. `None`.