Fix some writing issues in the docs (#14136)

* Fix some writing issues in the docs

* Run code quality check
This commit is contained in:
Reza Gharibi
2021-10-25 15:18:02 +03:30
committed by GitHub
parent 2ac65551ea
commit 3e04a41a9b
9 changed files with 28 additions and 28 deletions

View File

@@ -154,7 +154,7 @@ input elements was ``6.27e+04`` and same for the output was ``inf``.
You can see here, that ``T5DenseGatedGeluDense.forward`` resulted in output activations, whose absolute max value was
around 62.7K, which is very close to fp16's top limit of 64K. In the next frame we have ``Dropout`` which renormalizes
the weights, after it zeroed some of the elements, which pushes the absolute max value to more than 64K, and we get an
overlow (``inf``).
overflow (``inf``).
As you can see it's the previous frames that we need to look into when the numbers start going into very large for fp16
numbers.