From 9709ab116ca06bcbd60ef296354c726c5e45f402 Mon Sep 17 00:00:00 2001 From: Maria Khalusova Date: Wed, 13 Sep 2023 14:35:42 -0400 Subject: [PATCH] [docs] last hidden state vs hidden_states[-1] (#26142) * last hidden state clarification * feedback addressed --- docs/source/en/main_classes/output.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/docs/source/en/main_classes/output.md b/docs/source/en/main_classes/output.md index 93e1d78054..578b9e6542 100644 --- a/docs/source/en/main_classes/output.md +++ b/docs/source/en/main_classes/output.md @@ -40,6 +40,13 @@ an optional `attentions` attribute. Here we have the `loss` since we passed alon `hidden_states` and `attentions` because we didn't pass `output_hidden_states=True` or `output_attentions=True`. + + +When passing `output_hidden_states=True` you may expect the `outputs.hidden_states[-1]` to match `outputs.last_hidden_states` exactly. +However, this is not always the case. Some models apply normalization or subsequent process to the last hidden state when it's returned. + + + You can access each attribute as you would usually do, and if that attribute has not been returned by the model, you will get `None`. Here for instance `outputs.loss` is the loss computed by the model, and `outputs.attentions` is `None`.