From 906b638efadc2680fe1d4eb7f79e39cd459e462e Mon Sep 17 00:00:00 2001 From: thomwolf Date: Wed, 6 Mar 2019 10:24:19 +0100 Subject: [PATCH] updating readme --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index c9b8549843..b298938ca6 100644 --- a/README.md +++ b/README.md @@ -739,8 +739,8 @@ all_hidden_states = lower_hidden_states + [hidden_states] *Outputs* a tuple of (last_hidden_state, new_mems) - `softmax_output`: output of the (adaptive) softmax: - - if target is None: Negative log likelihood of shape [batch_size, sequence_length] - - else: log probabilities of tokens, shape [batch_size, sequence_length, n_tokens] + - if target is None: log probabilities of tokens, shape [batch_size, sequence_length, n_tokens] + - else: Negative log likelihood of target tokens with shape [batch_size, sequence_length] - `new_mems`: list (num layers) of updated mem states at the entry of each layer each mem state is a torch.FloatTensor of size [self.config.mem_len, batch_size, self.config.d_model]. Note that the first two dimensions are transposed in `mems` with regards to `input_ids`. #### 14. `GPT2Model`