fc6c8b0eaa
* Add run_mim_no_trainer.py draft from #20412 Add parse_args method and copy over other dependencies Add Method call for sending telemetry Initialize Accelerator Make one log on every process Set seed and Handle repository creation Initialize dataset and Set validation split Create Config Adapt Config Update Config Create Feature Extractor Create model Set column names Create transforms Create mask generator Create method to preprocess images Shuffle datasets if needed and set transforms Create Dataloaders Add optimizer Add learning rate scheduler Prepare everything with our accelerator Tie weights for TPU training Recalculate training steps and training epochs Set accelerator checkpointing steps Initialize trackers and store configuration Set total batch size Fix typo: mlm -> mim Log info at the start of training Load in the weights and states from previous save update the progress_bar if load from checkpoint Define train loop Add evaluation loop to training Add to parse_args method Push repo to hub Save accelerator state End training and save model and feature extractor Remove unused imports Fix trailing whitespace * Update code based on comments, Rename feature_extractor to image_processor * Fix linting * Add argument for learning rate * Add argument for setting number of training epochs * Remove incorrect logger argument * Convert max_train_steps to int for tqdm --------- Co-authored-by: Saad Mahmud <shuvro.mahmud79@gmail.com>
Examples
We host a wide range of example scripts for multiple learning frameworks. Simply choose your favorite: TensorFlow, PyTorch or JAX/Flax.
We also have some research projects, as well as some legacy examples. Note that unlike the main examples these are not actively maintained, and may require specific older versions of dependencies in order to run.
While we strive to present as many use cases as possible, the example scripts are just that - examples. It is expected that they won't work out-of-the box on your specific problem and that you will be required to change a few lines of code to adapt them to your needs. To help you with that, most of the examples fully expose the preprocessing of the data, allowing you to tweak and edit them as required.
Please discuss on the forum or in an issue a feature you would like to implement in an example before submitting a PR; we welcome bug fixes, but since we want to keep the examples as simple as possible it's unlikely that we will merge a pull request adding more functionality at the cost of readability.
Important note
Important
To make sure you can successfully run the latest versions of the example scripts, you have to install the library from source and install some example-specific requirements. To do this, execute the following steps in a new virtual environment:
git clone https://github.com/huggingface/transformers
cd transformers
pip install .
Then cd in the example folder of your choice and run
pip install -r requirements.txt
To browse the examples corresponding to released versions of 🤗 Transformers, click on the line below and then on your desired version of the library:
Examples for older versions of 🤗 Transformers
- v4.21.0
- v4.20.1
- v4.19.4
- v4.18.0
- v4.17.0
- v4.16.2
- v4.15.0
- v4.14.1
- v4.13.0
- v4.12.5
- v4.11.3
- v4.10.3
- v4.9.2
- v4.8.2
- v4.7.0
- v4.6.1
- v4.5.1
- v4.4.2
- v4.3.3
- v4.2.2
- v4.1.1
- v4.0.1
- v3.5.1
- v3.4.0
- v3.3.1
- v3.2.0
- v3.1.0
- v3.0.2
- v2.11.0
- v2.10.0
- v2.9.1
- v2.8.0
- v2.7.0
- v2.6.0
- v2.5.1
- v2.4.0
- v2.3.0
- v2.2.0
- v2.1.1
- v2.0.0
- v1.2.0
- v1.1.0
- v1.0.0
Alternatively, you can switch your cloned 🤗 Transformers to a specific version (for instance with v3.5.1) with
git checkout tags/v3.5.1
and run the example command as usual afterward.
Running the Examples on Remote Hardware with Auto-Setup
run_on_remote.py is a script that launches any example on remote self-hosted hardware, with automatic hardware and environment setup. It uses Runhouse to launch on self-hosted hardware (e.g. in your own cloud account or on-premise cluster) but there are other options for running remotely as well. You can easily customize the example used, command line arguments, dependencies, and type of compute hardware, and then run the script to automatically launch the example.
You can refer to hardware setup for more information about hardware and dependency setup with Runhouse, or this Colab tutorial for a more in-depth walkthrough.
You can run the script with the following commands:
# First install runhouse:
pip install runhouse
# For an on-demand V100 with whichever cloud provider you have configured:
python run_on_remote.py \
--example pytorch/text-generation/run_generation.py \
--model_type=gpt2 \
--model_name_or_path=gpt2 \
--prompt "I am a language model and"
# For byo (bring your own) cluster:
python run_on_remote.py --host <cluster_ip> --user <ssh_user> --key_path <ssh_key_path> \
--example <example> <args>
# For on-demand instances
python run_on_remote.py --instance <instance> --provider <provider> \
--example <example> <args>
You can also adapt the script to your own needs.