Convert model files from rst to mdx (#14865)
* First pass * Apply suggestions from code review * Apply suggestions from code review Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
This commit is contained in:
124
docs/source/model_doc/layoutlm.mdx
Normal file
124
docs/source/model_doc/layoutlm.mdx
Normal file
@@ -0,0 +1,124 @@
|
||||
<!--Copyright 2020 The HuggingFace Team. All rights reserved.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
|
||||
the License. You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
|
||||
an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
||||
specific language governing permissions and limitations under the License.
|
||||
-->
|
||||
|
||||
# LayoutLM
|
||||
|
||||
<a id='Overview'></a>
|
||||
|
||||
## Overview
|
||||
|
||||
The LayoutLM model was proposed in the paper [LayoutLM: Pre-training of Text and Layout for Document Image
|
||||
Understanding](https://arxiv.org/abs/1912.13318) by Yiheng Xu, Minghao Li, Lei Cui, Shaohan Huang, Furu Wei, and
|
||||
Ming Zhou. It's a simple but effective pretraining method of text and layout for document image understanding and
|
||||
information extraction tasks, such as form understanding and receipt understanding. It obtains state-of-the-art results
|
||||
on several downstream tasks:
|
||||
|
||||
- form understanding: the [FUNSD](https://guillaumejaume.github.io/FUNSD/) dataset (a collection of 199 annotated
|
||||
forms comprising more than 30,000 words).
|
||||
- receipt understanding: the [SROIE](https://rrc.cvc.uab.es/?ch=13) dataset (a collection of 626 receipts for
|
||||
training and 347 receipts for testing).
|
||||
- document image classification: the [RVL-CDIP](https://www.cs.cmu.edu/~aharley/rvl-cdip/) dataset (a collection of
|
||||
400,000 images belonging to one of 16 classes).
|
||||
|
||||
The abstract from the paper is the following:
|
||||
|
||||
*Pre-training techniques have been verified successfully in a variety of NLP tasks in recent years. Despite the
|
||||
widespread use of pretraining models for NLP applications, they almost exclusively focus on text-level manipulation,
|
||||
while neglecting layout and style information that is vital for document image understanding. In this paper, we propose
|
||||
the LayoutLM to jointly model interactions between text and layout information across scanned document images, which is
|
||||
beneficial for a great number of real-world document image understanding tasks such as information extraction from
|
||||
scanned documents. Furthermore, we also leverage image features to incorporate words' visual information into LayoutLM.
|
||||
To the best of our knowledge, this is the first time that text and layout are jointly learned in a single framework for
|
||||
document-level pretraining. It achieves new state-of-the-art results in several downstream tasks, including form
|
||||
understanding (from 70.72 to 79.27), receipt understanding (from 94.02 to 95.24) and document image classification
|
||||
(from 93.07 to 94.42).*
|
||||
|
||||
Tips:
|
||||
|
||||
- In addition to *input_ids*, [`~transformers.LayoutLMModel.forward`] also expects the input `bbox`, which are
|
||||
the bounding boxes (i.e. 2D-positions) of the input tokens. These can be obtained using an external OCR engine such
|
||||
as Google's [Tesseract](https://github.com/tesseract-ocr/tesseract) (there's a [Python wrapper](https://pypi.org/project/pytesseract/) available). Each bounding box should be in (x0, y0, x1, y1) format, where
|
||||
(x0, y0) corresponds to the position of the upper left corner in the bounding box, and (x1, y1) represents the
|
||||
position of the lower right corner. Note that one first needs to normalize the bounding boxes to be on a 0-1000
|
||||
scale. To normalize, you can use the following function:
|
||||
|
||||
```python
|
||||
def normalize_bbox(bbox, width, height):
|
||||
return [
|
||||
int(1000 * (bbox[0] / width)),
|
||||
int(1000 * (bbox[1] / height)),
|
||||
int(1000 * (bbox[2] / width)),
|
||||
int(1000 * (bbox[3] / height)),
|
||||
]
|
||||
```
|
||||
|
||||
Here, `width` and `height` correspond to the width and height of the original document in which the token
|
||||
occurs. Those can be obtained using the Python Image Library (PIL) library for example, as follows:
|
||||
|
||||
```python
|
||||
from PIL import Image
|
||||
|
||||
image = Image.open("name_of_your_document - can be a png file, pdf, etc.")
|
||||
|
||||
width, height = image.size
|
||||
```
|
||||
|
||||
- For a demo which shows how to fine-tune [`LayoutLMForTokenClassification`] on the [FUNSD dataset](https://guillaumejaume.github.io/FUNSD/) (a collection of annotated forms), see [this notebook](https://github.com/NielsRogge/Transformers-Tutorials/blob/master/LayoutLM/Fine_tuning_LayoutLMForTokenClassification_on_FUNSD.ipynb).
|
||||
It includes an inference part, which shows how to use Google's Tesseract on a new document.
|
||||
|
||||
This model was contributed by [liminghao1630](https://huggingface.co/liminghao1630). The original code can be found
|
||||
[here](https://github.com/microsoft/unilm/tree/master/layoutlm).
|
||||
|
||||
|
||||
## LayoutLMConfig
|
||||
|
||||
[[autodoc]] LayoutLMConfig
|
||||
|
||||
## LayoutLMTokenizer
|
||||
|
||||
[[autodoc]] LayoutLMTokenizer
|
||||
|
||||
## LayoutLMTokenizerFast
|
||||
|
||||
[[autodoc]] LayoutLMTokenizerFast
|
||||
|
||||
## LayoutLMModel
|
||||
|
||||
[[autodoc]] LayoutLMModel
|
||||
|
||||
## LayoutLMForMaskedLM
|
||||
|
||||
[[autodoc]] LayoutLMForMaskedLM
|
||||
|
||||
## LayoutLMForSequenceClassification
|
||||
|
||||
[[autodoc]] LayoutLMForSequenceClassification
|
||||
|
||||
## LayoutLMForTokenClassification
|
||||
|
||||
[[autodoc]] LayoutLMForTokenClassification
|
||||
|
||||
## TFLayoutLMModel
|
||||
|
||||
[[autodoc]] TFLayoutLMModel
|
||||
|
||||
## TFLayoutLMForMaskedLM
|
||||
|
||||
[[autodoc]] TFLayoutLMForMaskedLM
|
||||
|
||||
## TFLayoutLMForSequenceClassification
|
||||
|
||||
[[autodoc]] TFLayoutLMForSequenceClassification
|
||||
|
||||
## TFLayoutLMForTokenClassification
|
||||
|
||||
[[autodoc]] TFLayoutLMForTokenClassification
|
||||
Reference in New Issue
Block a user