From fe1c16e95a1c2f7730b5d7340b669a63ab0b8ded Mon Sep 17 00:00:00 2001 From: NielsRogge <48327001+NielsRogge@users.noreply.github.com> Date: Thu, 23 Nov 2023 18:44:08 +0100 Subject: [PATCH] [DPT, Dinov2] Add resources (#27655) * Add resources * Remove script * Update docs/source/en/model_doc/dinov2.md Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com> --------- Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com> --- docs/source/en/model_doc/dinov2.md | 12 ++++++++++++ docs/source/en/model_doc/dpt.md | 15 +++++++++++++++ 2 files changed, 27 insertions(+) diff --git a/docs/source/en/model_doc/dinov2.md b/docs/source/en/model_doc/dinov2.md index 72a0478924..dca9478677 100644 --- a/docs/source/en/model_doc/dinov2.md +++ b/docs/source/en/model_doc/dinov2.md @@ -55,6 +55,18 @@ with torch.no_grad(): print((last_hidden_states - traced_outputs[0]).abs().max()) ``` +## Resources + +A list of official Hugging Face and community (indicated by 🌎) resources to help you get started with DPT. + +- Demo notebooks for DINOv2 can be found [here](https://github.com/NielsRogge/Transformers-Tutorials/tree/master/DINOv2). 🌎 + + + +- [`Dinov2ForImageClassification`] is supported by this [example script](https://github.com/huggingface/transformers/tree/main/examples/pytorch/image-classification) and [notebook](https://colab.research.google.com/github/huggingface/notebooks/blob/main/examples/image_classification.ipynb). +- See also: [Image classification task guide](../tasks/image_classification) + +If you're interested in submitting a resource to be included here, please feel free to open a Pull Request and we'll review it! The resource should ideally demonstrate something new instead of duplicating an existing resource. ## Dinov2Config diff --git a/docs/source/en/model_doc/dpt.md b/docs/source/en/model_doc/dpt.md index 5e3e25343c..a02313a312 100644 --- a/docs/source/en/model_doc/dpt.md +++ b/docs/source/en/model_doc/dpt.md @@ -32,6 +32,21 @@ alt="drawing" width="600"/> This model was contributed by [nielsr](https://huggingface.co/nielsr). The original code can be found [here](https://github.com/isl-org/DPT). +## Usage tips + +DPT is compatible with the [`AutoBackbone`] class. This allows to use the DPT framework with various computer vision backbones available in the library, such as [`VitDetBackbone`] or [`Dinov2Backbone`]. One can create it as follows: + +```python +from transformers import Dinov2Config, DPTConfig, DPTForDepthEstimation + +# initialize with a Transformer-based backbone such as DINOv2 +# in that case, we also specify `reshape_hidden_states=False` to get feature maps of shape (batch_size, num_channels, height, width) +backbone_config = Dinov2Config.from_pretrained("facebook/dinov2-base", out_features=["stage1", "stage2", "stage3", "stage4"], reshape_hidden_states=False) + +config = DPTConfig(backbone_config=backbone_config) +model = DPTForDepthEstimation(config=config) +``` + ## Resources A list of official Hugging Face and community (indicated by 🌎) resources to help you get started with DPT.