SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity

TF ALBERT + TF Utilities + Fix warnings

Browse Source
This commit is contained in:
Lysandre
2020-01-15 15:50:30 -05:00
committed by Lysandre Debut
parent 00df3d4de0
commit 3922a2497e
6 changed files with 148 additions and 126 deletions
Download Patch File Download Diff File Expand all files Collapse all files

7
docs/source/main_classes/optimizer_schedules.rst
View File

@@ -20,14 +20,12 @@ The ``.optimization`` module provides:
:members:
.. autofunction:: transformers.create_optimizer
:members:
Schedules
----------------------------------------------------
Learning Rate Schedules
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. autofunction:: transformers.get_constant_schedule
@@ -39,7 +37,6 @@ Learning Rate Schedules
.. autofunction:: transformers.get_cosine_schedule_with_warmup
:members:
.. image:: /imgs/warmup_cosine_schedule.png
:target: /imgs/warmup_cosine_schedule.png
@@ -63,7 +60,7 @@ Learning Rate Schedules
``Warmup``
~~~~~~~~~~~~~~~~
.. autoclass:: transformers.Warmup
.. autoclass:: transformers.WarmUp
:members:
Gradient Strategies