SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity

Updating documentation for converting tensorflow model to reflect the new cli convert format.

Browse Source 
Signed-off-by: Morgan Funtowicz <morgan@huggingface.co>
This commit is contained in:
Morgan Funtowicz
2020-01-02 12:07:39 +01:00
parent 629b22adcf
commit 80faf22b4a
1 changed files with 37 additions and 25 deletions
Download Patch File Download Diff File Expand all files Collapse all files

62
docs/source/converting_tensorflow_models.rst
View File

@@ -3,6 +3,12 @@ Converting Tensorflow Checkpoints
A command-line interface is provided to convert original Bert/GPT/GPT-2/Transformer-XL/XLNet/XLM checkpoints in models than be loaded using the ``from_pretrained`` methods of the library. A command-line interface is provided to convert original Bert/GPT/GPT-2/Transformer-XL/XLNet/XLM checkpoints in models than be loaded using the ``from_pretrained`` methods of the library.
.. note::
Since 2.3.0 the conversion script is now part of the transformers CLI (**transformers-cli**)
available in any transformers >= 2.3.0 installation.
The documentation below reflects the **transformers-cli convert** command format.
BERT BERT
^^^^ ^^^^
@@ -20,10 +26,10 @@ Here is an example of the conversion process for a pre-trained ``BERT-Base Uncas
export BERT_BASE_DIR=/path/to/bert/uncased_L-12_H-768_A-12 export BERT_BASE_DIR=/path/to/bert/uncased_L-12_H-768_A-12
transformers bert \ transformers-cli convert --model_type bert \
$BERT_BASE_DIR/bert_model.ckpt \ --tf_checkpoint $BERT_BASE_DIR/bert_model.ckpt \
$BERT_BASE_DIR/bert_config.json \ --config $BERT_BASE_DIR/bert_config.json \
$BERT_BASE_DIR/pytorch_model.bin --pytorch_dump_output $BERT_BASE_DIR/pytorch_model.bin
You can download Google's pre-trained models for the conversion `here <https://github.com/google-research/bert#pre-trained-models>`__. You can download Google's pre-trained models for the conversion `here <https://github.com/google-research/bert#pre-trained-models>`__.
@@ -36,10 +42,12 @@ Here is an example of the conversion process for a pre-trained OpenAI GPT model,
export OPENAI_GPT_CHECKPOINT_FOLDER_PATH=/path/to/openai/pretrained/numpy/weights export OPENAI_GPT_CHECKPOINT_FOLDER_PATH=/path/to/openai/pretrained/numpy/weights
transformers gpt \ transformers-cli convert --model_type gpt \
$OPENAI_GPT_CHECKPOINT_FOLDER_PATH \ --tf_checkpoint $OPENAI_GPT_CHECKPOINT_FOLDER_PATH \
$PYTORCH_DUMP_OUTPUT \ --pytorch_dump_output $PYTORCH_DUMP_OUTPUT \
[OPENAI_GPT_CONFIG] [--config OPENAI_GPT_CONFIG] \
[--finetuning_task_name OPENAI_GPT_FINETUNED_TASK] \
OpenAI GPT-2 OpenAI GPT-2
^^^^^^^^^^^^ ^^^^^^^^^^^^
@@ -50,10 +58,11 @@ Here is an example of the conversion process for a pre-trained OpenAI GPT-2 mode
export OPENAI_GPT2_CHECKPOINT_PATH=/path/to/gpt2/pretrained/weights export OPENAI_GPT2_CHECKPOINT_PATH=/path/to/gpt2/pretrained/weights
transformers gpt2 \ transformers-cli convert --model_type gpt2 \
$OPENAI_GPT2_CHECKPOINT_PATH \ --tf_checkpoint $OPENAI_GPT2_CHECKPOINT_PATH \
$PYTORCH_DUMP_OUTPUT \ --pytorch_dump_output $PYTORCH_DUMP_OUTPUT \
[OPENAI_GPT2_CONFIG] [--config OPENAI_GPT2_CONFIG] \
[--finetuning_task_name OPENAI_GPT2_FINETUNED_TASK]
Transformer-XL Transformer-XL
^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^
@@ -64,27 +73,28 @@ Here is an example of the conversion process for a pre-trained Transformer-XL mo
export TRANSFO_XL_CHECKPOINT_FOLDER_PATH=/path/to/transfo/xl/checkpoint export TRANSFO_XL_CHECKPOINT_FOLDER_PATH=/path/to/transfo/xl/checkpoint
transformers transfo_xl \ transformers-cli convert --model_type transfo_xl \
$TRANSFO_XL_CHECKPOINT_FOLDER_PATH \ --tf_checkpoint $TRANSFO_XL_CHECKPOINT_FOLDER_PATH \
$PYTORCH_DUMP_OUTPUT \ --pytorch_dump_output $PYTORCH_DUMP_OUTPUT \
[TRANSFO_XL_CONFIG] [--config TRANSFO_XL_CONFIG] \
[--finetuning_task_name TRANSFO_XL_FINETUNED_TASK]
XLNet XLNet
^^^^^ ^^^^^
Here is an example of the conversion process for a pre-trained XLNet model, fine-tuned on STS-B using the TensorFlow script: Here is an example of the conversion process for a pre-trained XLNet model:
.. code-block:: shell .. code-block:: shell
export TRANSFO_XL_CHECKPOINT_PATH=/path/to/xlnet/checkpoint export TRANSFO_XL_CHECKPOINT_PATH=/path/to/xlnet/checkpoint
export TRANSFO_XL_CONFIG_PATH=/path/to/xlnet/config export TRANSFO_XL_CONFIG_PATH=/path/to/xlnet/config
transformers xlnet \ transformers-cli convert --model_type xlnet \
$TRANSFO_XL_CHECKPOINT_PATH \ --tf_checkpoint $TRANSFO_XL_CHECKPOINT_PATH \
$TRANSFO_XL_CONFIG_PATH \ --config $TRANSFO_XL_CONFIG_PATH \
$PYTORCH_DUMP_OUTPUT \ --pytorch_dump_output $PYTORCH_DUMP_OUTPUT \
STS-B \ [--finetuning_task_name XLNET_FINETUNED_TASK] \
XLM XLM
@@ -96,6 +106,8 @@ Here is an example of the conversion process for a pre-trained XLM model:
export XLM_CHECKPOINT_PATH=/path/to/xlm/checkpoint export XLM_CHECKPOINT_PATH=/path/to/xlm/checkpoint
transformers xlm \ transformers-cli convert --model_type xlm \
$XLM_CHECKPOINT_PATH \ --tf_checkpoint $XLM_CHECKPOINT_PATH \
$PYTORCH_DUMP_OUTPUT \ --pytorch_dump_output $PYTORCH_DUMP_OUTPUT
[--config XML_CONFIG] \
[--finetuning_task_name XML_FINETUNED_TASK]