From d2e5b19b821f0cf43c7cf4f01be5faa1cb20aa64 Mon Sep 17 00:00:00 2001
From: Arthur <48595927+ArthurZucker@users.noreply.github.com>
Date: Mon, 17 Oct 2022 11:23:20 +0200
Subject: [PATCH] Add doctest info in testingmdx (#19623)

---
 docs/source/en/testing.mdx | 41 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 41 insertions(+)

diff --git a/docs/source/en/testing.mdx b/docs/source/en/testing.mdx
index 23c0be7f1a..e60c285ca5 100644
--- a/docs/source/en/testing.mdx
+++ b/docs/source/en/testing.mdx
@@ -176,6 +176,47 @@ If you want to include only tests that include both patterns, `and` is to be use
 ```bash
 pytest -k "test and ada" tests/test_optimization.py
 ```
+### Run documentation tests 
+
+In order to test whether the documentation examples are correct, you should checkt that the `doctests` are passing. 
+As an example, let's use [`WhisperModel.forward`'s docstring](https://github.com/huggingface/transformers/blob/main/src/transformers/models/whisper/modeling_whisper.py#L1017-L1035): 
+
+```python 
+r"""
+Returns:
+
+Example:
+    ```python
+    >>> import torch
+    >>> from transformers import WhisperModel, WhisperFeatureExtractor
+    >>> from datasets import load_dataset
+
+    >>> model = WhisperModel.from_pretrained("openai/whisper-base")
+    >>> feature_extractor = WhisperFeatureExtractor.from_pretrained("openai/whisper-base")
+    >>> ds = load_dataset("hf-internal-testing/librispeech_asr_dummy", "clean", split="validation")
+    >>> inputs = feature_extractor(ds[0]["audio"]["array"], return_tensors="pt")
+    >>> input_features = inputs.input_features
+    >>> decoder_input_ids = torch.tensor([[1, 1]]) * model.config.decoder_start_token_id
+    >>> last_hidden_state = model(input_features, decoder_input_ids=decoder_input_ids).last_hidden_state
+    >>> list(last_hidden_state.shape)
+    [1, 2, 512]
+    ```"""
+
+```
+3 steps are required to debug the docstring examples : 
+1. In order to properly run the test, **an extra line has to be added** at the end of the docstring. This can be automatically done on any file using : 
+```bash 
+python utils/prepare_for_doc_test.py <path_to_file_or_dir>
+```
+
+2. Then, you can use the following line to automatically test every docstring example in the desired file : 
+```bash 
+pytest --doctest-modules <path_to_file_or_dir>
+```
+3. Once you are done debugging, you need to remove the extra line added in step **1.** by running the follwing : 
+```bash 
+python utils/prepare_for_doc_test.py <path_to_file_or_dir> --remove_new_line
+```
 
 ### Run only modified tests