From 0be5f4a00c0fda64110e18eaadcd9d321bfebd9d Mon Sep 17 00:00:00 2001
From: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Date: Thu, 24 Sep 2020 04:34:18 -0400
Subject: [PATCH] Expand a bit the documentation doc (#7350)

---
 docs/README.md | 65 +++++++++++++++++++++++++++++++++++++-------------
 1 file changed, 49 insertions(+), 16 deletions(-)

diff --git a/docs/README.md b/docs/README.md
index 6da2f78f3a..0c011ad1db 100644
--- a/docs/README.md
+++ b/docs/README.md
@@ -88,20 +88,25 @@ The `huggingface/transformers` documentation follows the
 [Google documentation](https://sphinxcontrib-napoleon.readthedocs.io/en/latest/example_google.html) style. It is
 mostly written in ReStructuredText 
 ([Sphinx simple documentation](https://www.sphinx-doc.org/en/master/usage/restructuredtext/index.html), 
-[Sourceforge complete documentation](https://docutils.sourceforge.io/docs/ref/rst/restructuredtext.html))
+[Sourceforge complete documentation](https://docutils.sourceforge.io/docs/ref/rst/restructuredtext.html)).
 
-### Adding a new section
 
-A section is a page held in the `Notes` toc-tree on the documentation. Adding a new section is done in two steps:
+### Adding a new tutorial
+
+Adding a new tutorial or section is done in two steps:
 
 - Add a new file under `./source`. This file can either be ReStructuredText (.rst) or Markdown (.md).
 - Link that file in `./source/index.rst` on the correct toc-tree.
 
+Make sure to put your new file under the proper section. It's unlikely to go in the first section (*Get Started*), so
+depending on the intended targets (beginners, more advanced users or researchers) it should go in section two, three or
+four.
+
 ### Adding a new model
 
 When adding a new model:
  
-- Create a file `xxx.rst` under `./source/model_doc`. 
+- Create a file `xxx.rst` under `./source/model_doc` (don't hesitate to copy an existing file as template). 
 - Link that file in `./source/index.rst` on the `model_doc` toc-tree.
 - Write a short overview of the model:
     - Overview with paper & authors
@@ -120,18 +125,18 @@ When adding a new model:
 These classes should be added using the RST syntax. Usually as follows:
 ```
 XXXConfig
-~~~~~~~~~~~~~~~~~~~~~
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 .. autoclass:: transformers.XXXConfig
     :members:
 ```
 
-This will include every public method of the configuration. If for some reason you wish for a method not to be
-displayed in the documentation, you can do so by specifying which methods should be in the docs:
+This will include every public method of the configuration that is documented. If for some reason you wish for a method
+not to be displayed in the documentation, you can do so by specifying which methods should be in the docs:
 
 ```
 XXXTokenizer
-~~~~~~~~~~~~~~~~~~~~~
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 .. autoclass:: transformers.XXXTokenizer
     :members: build_inputs_with_special_tokens, get_special_tokens_mask,
@@ -142,13 +147,17 @@ XXXTokenizer
 ### Writing source documentation
 
 Values that should be put in `code` should either be surrounded by double backticks: \`\`like so\`\` or be written as
-an object using the :obj: syntax: :obj:\`like so\`.
+an object using the :obj: syntax: :obj:\`like so\`. Note that argument names and objects like True, None or any strings
+should usually be put in `code`.
 
 When mentionning a class, it is recommended to use the :class: syntax as the mentioned class will be automatically
-linked by Sphinx: :class:\`transformers.XXXClass\`
+linked by Sphinx: :class:\`~transformers.XXXClass\`
 
-When mentioning a function, it is recommended to use the :func: syntax as the mentioned method will be automatically
-linked by Sphinx: :func:\`transformers.XXXClass.method\`
+When mentioning a function, it is recommended to use the :func: syntax as the mentioned function will be automatically
+linked by Sphinx: :func:\`~transformers.function\`.
+
+When mentioning a method, it is recommended to use the :meth: syntax as the mentioned method will be automatically
+linked by Sphinx: :meth:\`~transformers.XXXClass.method\`.
 
 Links should be done as so (note the double underscore at the end): \`text for the link <./local-link-or-global-link#loc>\`__
 
@@ -165,13 +174,34 @@ Here's an example showcasing everything so far:
         input_ids (:obj:`torch.LongTensor` of shape :obj:`(batch_size, sequence_length)`):
             Indices of input sequence tokens in the vocabulary.
 
-            Indices can be obtained using :class:`transformers.AlbertTokenizer`.
-            See :func:`transformers.PreTrainedTokenizer.encode` and
-            :func:`transformers.PreTrainedTokenizer.__call__` for details.
+            Indices can be obtained using :class:`~transformers.AlbertTokenizer`.
+            See :meth:`~transformers.PreTrainedTokenizer.encode` and
+            :meth:`~transformers.PreTrainedTokenizer.__call__` for details.
 
             `What are input IDs? <../glossary.html#input-ids>`__
 ```
 
+For optional arguments or arguments with defaults we follow the following syntax: imagine we have a function with the
+following signature:
+
+```
+def my_function(x: str = None, a: float = 1):
+```
+
+then its documentation should look like this:
+
+```
+    Args:
+        x (:obj:`str`, `optional`):
+            This argument controls ...
+        a (:obj:`float`, `optional`, defaults to 1):
+            This argument is used to ...
+```
+
+Note that we always omit the "defaults to :obj:\`None\`" when None is the default for any argument. Also note that even
+if the first line describing your argument type and its default gets long, you can't break it on several lines. You can
+however write as many lines as you want in the indented description (see the example above with `input_ids`). 
+
 #### Writing a multi-line code block 
 
 Multi-line code blocks can be useful for displaying examples. They are done like so:
@@ -186,6 +216,9 @@ Example::
 
 The `Example` string at the beginning can be replaced by anything as long as there are two semicolons following it.
 
+We follow the [doctest](https://docs.python.org/3/library/doctest.html) syntax for the examples to automatically test
+the results stay consistent with the library.
+
 #### Writing a return block
 
 Arguments should be defined with the `Args:` prefix, followed by a line return and an indentation. 
@@ -207,5 +240,5 @@ Here's an example for a single value return:
 
 ```
     Returns:
-        A list of integers in the range [0, 1]: 1 for a special token, 0 for a sequence token.
+        :obj:`List[int]`: A list of integers in the range [0, 1] --- 1 for a special token, 0 for a sequence token.
 ```