Enable doc in Spanish (#16518)
* Reorganize doc for multilingual support * Fix style * Style * Toc trees * Adapt templates
This commit is contained in:
110
docs/source/en/model_doc/data2vec.mdx
Normal file
110
docs/source/en/model_doc/data2vec.mdx
Normal file
@@ -0,0 +1,110 @@
|
||||
<!--Copyright 2022 The HuggingFace Team. All rights reserved.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
|
||||
the License. You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
|
||||
an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
||||
specific language governing permissions and limitations under the License.
|
||||
-->
|
||||
|
||||
# Data2Vec
|
||||
|
||||
## Overview
|
||||
|
||||
The Data2Vec model was proposed in [data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Language](https://arxiv.org/pdf/2202.03555) by Alexei Baevski, Wei-Ning Hsu, Qiantong Xu, Arun Babu, Jiatao Gu and Michael Auli.
|
||||
Data2Vec proposes a unified framework for self-supervised learning across different data modalities - text, audio and images.
|
||||
Importantly, predicted targets for pre-training are contextualized latent representations of the inputs, rather than modality-specific, context-independent targets.
|
||||
|
||||
The abstract from the paper is the following:
|
||||
|
||||
*While the general idea of self-supervised learning is identical across modalities, the actual algorithms and
|
||||
objectives differ widely because they were developed with a single modality in mind. To get us closer to general
|
||||
self-supervised learning, we present data2vec, a framework that uses the same learning method for either speech,
|
||||
NLP or computer vision. The core idea is to predict latent representations of the full input data based on a
|
||||
masked view of the input in a selfdistillation setup using a standard Transformer architecture.
|
||||
Instead of predicting modality-specific targets such as words, visual tokens or units of human speech which
|
||||
are local in nature, data2vec predicts contextualized latent representations that contain information from
|
||||
the entire input. Experiments on the major benchmarks of speech recognition, image classification, and
|
||||
natural language understanding demonstrate a new state of the art or competitive performance to predominant approaches.
|
||||
Models and code are available at www.github.com/pytorch/fairseq/tree/master/examples/data2vec.*
|
||||
|
||||
Tips:
|
||||
|
||||
- Both Data2VecAudio and Data2VecText have been trained using the same self-supervised learning method.
|
||||
In the case of Data2VecAudio, preprocessing is identical to [`RobertaModel`], including tokenization.
|
||||
|
||||
This model was contributed by [edugp](https://huggingface.co/edugp).
|
||||
The original code can be found [here](https://github.com/pytorch/fairseq/tree/main/examples/data2vec).
|
||||
|
||||
|
||||
## Data2VecTextConfig
|
||||
|
||||
[[autodoc]] Data2VecTextConfig
|
||||
|
||||
## Data2VecAudioConfig
|
||||
|
||||
[[autodoc]] Data2VecAudioConfig
|
||||
|
||||
## Data2VecAudioModel
|
||||
|
||||
[[autodoc]] Data2VecAudioModel
|
||||
- forward
|
||||
|
||||
|
||||
## Data2VecAudioForAudioFrameClassification
|
||||
|
||||
[[autodoc]] Data2VecAudioForAudioFrameClassification
|
||||
- forward
|
||||
|
||||
## Data2VecAudioForCTC
|
||||
|
||||
[[autodoc]] Data2VecAudioForCTC
|
||||
- forward
|
||||
|
||||
## Data2VecAudioForSequenceClassification
|
||||
|
||||
[[autodoc]] Data2VecAudioForSequenceClassification
|
||||
- forward
|
||||
|
||||
## Data2VecAudioForXVector
|
||||
|
||||
[[autodoc]] Data2VecAudioForXVector
|
||||
- forward
|
||||
|
||||
## Data2VecTextModel
|
||||
|
||||
[[autodoc]] Data2VecTextModel
|
||||
- forward
|
||||
|
||||
## Data2VecTextForCausalLM
|
||||
|
||||
[[autodoc]] Data2VecTextForCausalLM
|
||||
- forward
|
||||
|
||||
## Data2VecTextForMaskedLM
|
||||
|
||||
[[autodoc]] Data2VecTextForMaskedLM
|
||||
- forward
|
||||
|
||||
## Data2VecTextForSequenceClassification
|
||||
|
||||
[[autodoc]] Data2VecTextForSequenceClassification
|
||||
- forward
|
||||
|
||||
## Data2VecTextForMultipleChoice
|
||||
|
||||
[[autodoc]] Data2VecTextForMultipleChoice
|
||||
- forward
|
||||
|
||||
## Data2VecTextForTokenClassification
|
||||
|
||||
[[autodoc]] Data2VecTextForTokenClassification
|
||||
- forward
|
||||
|
||||
## Data2VecTextForQuestionAnswering
|
||||
|
||||
[[autodoc]] Data2VecTextForQuestionAnswering
|
||||
- forward
|
||||
Reference in New Issue
Block a user