SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity

[docs] Trainer docs (#28145)

Browse Source 
* fsdp, debugging, gpu selection

* fix hfoption

* fix
This commit is contained in:
Steven Liu
2023-12-20 10:37:23 -08:00
committed by GitHub
parent ee298a16a2
commit 01c081d138
6 changed files with 297 additions and 346 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
docs/source/en/trainer.md
View File

@@ -256,6 +256,12 @@ NEFTune is disabled after training to restore the original embedding layer to av
The [`Trainer`] class is powered by [Accelerate](https://hf.co/docs/accelerate), a library for easily training PyTorch models in distributed environments with support for integrations such as [FullyShardedDataParallel (FSDP)](https://pytorch.org/blog/introducing-pytorch-fully-sharded-data-parallel-api/) and [DeepSpeed](https://www.deepspeed.ai/).
<Tip>
Learn more about FSDP sharding strategies, CPU offloading, and more with the [`Trainer`] in the [Fully Sharded Data Parallel](fsdp) guide.
</Tip>
To use Accelerate with [`Trainer`], run the [`accelerate.config`](https://huggingface.co/docs/accelerate/package_reference/cli#accelerate-config) command to set up training for your training environment. This command creates a `config_file.yaml` that'll be used when you launch your training script. For example, some example configurations you can setup are:
<hfoptions id="config">