SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity

[SuperPoint, PaliGemma] Update docs (#31025)

Browse Source 
* Update docs

* Add PaliGemma resources

* Address comment

* Update docs
This commit is contained in:
NielsRogge
2024-05-28 13:22:06 +02:00
committed by GitHub
parent 66add161dc
commit 90da0b1c9f
2 changed files with 57 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files

21
docs/source/en/model_doc/superpoint.md
View File

@@ -38,12 +38,17 @@ to repeatedly detect a much richer set of interest points than the initial pre-a
traditional corner detector. The final system gives rise to state-of-the-art homography estimation results on HPatches
when compared to LIFT, SIFT and ORB.*
## How to use
<img src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/model_doc/superpoint_architecture.png"
alt="drawing" width="500"/>
<small> SuperPoint overview. Taken from the <a href="https://arxiv.org/abs/1712.07629v4">original paper.</a> </small>
## Usage tips
Here is a quick example of using the model to detect interest points in an image:
```python
from transformers import AutoImageProcessor, AutoModel
from transformers import AutoImageProcessor, SuperPointForKeypointDetection
import torch
from PIL import Image
import requests
@@ -52,7 +57,7 @@ url = "http://images.cocodataset.org/val2017/000000039769.jpg"
image = Image.open(requests.get(url, stream=True).raw)
processor = AutoImageProcessor.from_pretrained("magic-leap-community/superpoint")
model = AutoModel.from_pretrained("magic-leap-community/superpoint")
model = SuperPointForKeypointDetection.from_pretrained("magic-leap-community/superpoint")
inputs = processor(image, return_tensors="pt")
outputs = model(**inputs)
@@ -64,7 +69,7 @@ You can also feed multiple images to the model. Due to the nature of SuperPoint,
you will need to use the mask attribute to retrieve the respective information :
```python
from transformers import AutoImageProcessor, AutoModel
from transformers import AutoImageProcessor, SuperPointForKeypointDetection
import torch
from PIL import Image
import requests
@@ -77,7 +82,7 @@ image_2 = Image.open(requests.get(url_image_2, stream=True).raw)
images = [image_1, image_2]
processor = AutoImageProcessor.from_pretrained("magic-leap-community/superpoint")
model = AutoModel.from_pretrained("magic-leap-community/superpoint")
model = SuperPointForKeypointDetection.from_pretrained("magic-leap-community/superpoint")
inputs = processor(images, return_tensors="pt")
outputs = model(**inputs)
@@ -103,6 +108,12 @@ cv2.imwrite("output_image.png", image)
This model was contributed by [stevenbucaille](https://huggingface.co/stevenbucaille).
The original code can be found [here](https://github.com/magicleap/SuperPointPretrainedNetwork).
## Resources
A list of official Hugging Face and community (indicated by 🌎) resources to help you get started with SuperPoint. If you're interested in submitting a resource to be included here, please feel free to open a Pull Request and we'll review it! The resource should ideally demonstrate something new instead of duplicating an existing resource.
- A notebook showcasing inference and visualization with SuperPoint can be found [here](https://github.com/NielsRogge/Transformers-Tutorials/blob/master/SuperPoint/Inference_with_SuperPoint_to_detect_interest_points_in_an_image.ipynb). 🌎
## SuperPointConfig
[[autodoc]] SuperPointConfig