Remove outdated BERT tips (#6217)

* Remove out-dated BERT tips

* Update modeling_outputs.py

* Update bert.rst

* Update bert.rst
This commit is contained in:
Kevin Canwen Xu
2020-08-04 01:17:56 +08:00
committed by GitHub
parent e4920c92d6
commit 3c289fb38c
2 changed files with 2 additions and 11 deletions

View File

@@ -45,10 +45,6 @@ class BaseModelOutputWithPooling(ModelOutput):
further processed by a Linear layer and a Tanh activation function. The Linear
layer weights are trained from the next sentence prediction (classification)
objective during pretraining.
This output is usually *not* a good summary
of the semantic content of the input, you're often better with averaging or pooling
the sequence of hidden-states for the whole input sequence.
hidden_states (:obj:`tuple(torch.FloatTensor)`, `optional`, returned when ``output_hidden_states=True`` is passed or when ``config.output_hidden_states=True``):
Tuple of :obj:`torch.FloatTensor` (one for the output of the embeddings + one for the output of each layer)
of shape :obj:`(batch_size, sequence_length, hidden_size)`.