From 692c3c6b73b8d4cb312950f60a05ab8ad37eff04 Mon Sep 17 00:00:00 2001 From: Matt Date: Mon, 22 Jan 2024 13:46:04 +0000 Subject: [PATCH] Add config tip to custom model docs (#28601) Add tip to custom model docs --- docs/source/en/custom_models.md | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/docs/source/en/custom_models.md b/docs/source/en/custom_models.md index 22ba58b9d9..c64b2af5c2 100644 --- a/docs/source/en/custom_models.md +++ b/docs/source/en/custom_models.md @@ -34,6 +34,16 @@ Before we dive into the model, let's first write its configuration. The configur will contain all the necessary information to build the model. As we will see in the next section, the model can only take a `config` to be initialized, so we really need that object to be as complete as possible. + + +Models in the `transformers` library itself generally follow the convention that they accept a `config` object +in their `__init__` method, and then pass the whole `config` to sub-layers in the model, rather than breaking the +config object into multiple arguments that are all passed individually to sub-layers. Writing your model in this +style results in simpler code with a clear "source of truth" for any hyperparameters, and also makes it easier +to reuse code from other models in `transformers`. + + + In our example, we will take a couple of arguments of the ResNet class that we might want to tweak. Different configurations will then give us the different types of ResNets that are possible. We then just store those arguments, after checking the validity of a few of them.