translate internal folder files to chinese (#27638)
* translate * update * update --------- Co-authored-by: jiaqiw <wangjiaqi50@huawei.com>
This commit is contained in:
jiaqiw09
@@ -40,7 +40,7 @@ Most of those are only useful if you are studying the code of the Trainer in the
|
|||||||
|
|
||||||
[[autodoc]] trainer_pt_utils.DistributedTensorGatherer
|
[[autodoc]] trainer_pt_utils.DistributedTensorGatherer
|
||||||
|
|
||||||
## Distributed Evaluation
|
## Trainer Argument Parser
|
||||||
|
|
||||||
[[autodoc]] HfArgumentParser
|
[[autodoc]] HfArgumentParser
|
||||||
|
|
||||||
|
|||||||
@@ -107,6 +107,26 @@
|
|||||||
- local: main_classes/image_processor
|
- local: main_classes/image_processor
|
||||||
title: Image Processor
|
title: Image Processor
|
||||||
title: 主要类
|
title: 主要类
|
||||||
|
- sections:
|
||||||
|
- local: internal/modeling_utils
|
||||||
|
title: 自定义层和工具
|
||||||
|
- local: internal/pipelines_utils
|
||||||
|
title: pipelines工具
|
||||||
|
- local: internal/tokenization_utils
|
||||||
|
title: Tokenizers工具
|
||||||
|
- local: internal/trainer_utils
|
||||||
|
title: 训练器工具
|
||||||
|
- local: internal/generation_utils
|
||||||
|
title: 生成工具
|
||||||
|
- local: internal/image_processing_utils
|
||||||
|
title: 图像处理工具
|
||||||
|
- local: internal/audio_utils
|
||||||
|
title: 音频处理工具
|
||||||
|
- local: internal/file_utils
|
||||||
|
title: 通用工具
|
||||||
|
- local: internal/time_series_utils
|
||||||
|
title: 时序数据工具
|
||||||
|
title: 内部辅助工具
|
||||||
title: 应用程序接口 (API)
|
title: 应用程序接口 (API)
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
40
docs/source/zh/internal/audio_utils.md
Normal file
40
docs/source/zh/internal/audio_utils.md
Normal file
@@ -0,0 +1,40 @@
|
|||||||
|
<!--Copyright 2023 The HuggingFace Team. All rights reserved.
|
||||||
|
|
||||||
|
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
|
||||||
|
the License. You may obtain a copy of the License at
|
||||||
|
|
||||||
|
http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
|
||||||
|
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
|
||||||
|
an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
||||||
|
specific language governing permissions and limitations under the License.
|
||||||
|
|
||||||
|
⚠️ Note that this file is in Markdown but contain specific syntax for our doc-builder (similar to MDX) that may not be
|
||||||
|
rendered properly in your Markdown viewer.
|
||||||
|
|
||||||
|
-->
|
||||||
|
|
||||||
|
# `FeatureExtractors`的工具
|
||||||
|
|
||||||
|
此页面列出了音频 [`FeatureExtractor`] 可以使用的所有实用函数，以便使用常见的算法（如 *Short Time Fourier Transform* 或 *log mel spectrogram*）从原始音频中计算特殊特征。
|
||||||
|
|
||||||
|
其中大多数仅在您研究库中音频processors的代码时有用。
|
||||||
|
|
||||||
|
|
||||||
|
## 音频转换
|
||||||
|
|
||||||
|
[[autodoc]] audio_utils.hertz_to_mel
|
||||||
|
|
||||||
|
[[autodoc]] audio_utils.mel_to_hertz
|
||||||
|
|
||||||
|
[[autodoc]] audio_utils.mel_filter_bank
|
||||||
|
|
||||||
|
[[autodoc]] audio_utils.optimal_fft_length
|
||||||
|
|
||||||
|
[[autodoc]] audio_utils.window_function
|
||||||
|
|
||||||
|
[[autodoc]] audio_utils.spectrogram
|
||||||
|
|
||||||
|
[[autodoc]] audio_utils.power_to_db
|
||||||
|
|
||||||
|
[[autodoc]] audio_utils.amplitude_to_db
|
||||||
50
docs/source/zh/internal/file_utils.md
Normal file
50
docs/source/zh/internal/file_utils.md
Normal file
@@ -0,0 +1,50 @@
|
|||||||
|
<!--Copyright 2021 The HuggingFace Team. All rights reserved.
|
||||||
|
|
||||||
|
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
|
||||||
|
the License. You may obtain a copy of the License at
|
||||||
|
|
||||||
|
http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
|
||||||
|
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
|
||||||
|
an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
||||||
|
specific language governing permissions and limitations under the License.
|
||||||
|
|
||||||
|
⚠️ Note that this file is in Markdown but contain specific syntax for our doc-builder (similar to MDX) that may not be
|
||||||
|
rendered properly in your Markdown viewer.
|
||||||
|
|
||||||
|
-->
|
||||||
|
|
||||||
|
# 通用工具
|
||||||
|
|
||||||
|
此页面列出了在`utils.py`文件中找到的所有Transformers通用实用函数。
|
||||||
|
|
||||||
|
其中大多数仅在您研究库中的通用代码时才有用。
|
||||||
|
|
||||||
|
|
||||||
|
## Enums和namedtuples(命名元组)
|
||||||
|
|
||||||
|
[[autodoc]] utils.ExplicitEnum
|
||||||
|
|
||||||
|
[[autodoc]] utils.PaddingStrategy
|
||||||
|
|
||||||
|
[[autodoc]] utils.TensorType
|
||||||
|
|
||||||
|
## 特殊的装饰函数
|
||||||
|
|
||||||
|
[[autodoc]] utils.add_start_docstrings
|
||||||
|
|
||||||
|
[[autodoc]] utils.add_start_docstrings_to_model_forward
|
||||||
|
|
||||||
|
[[autodoc]] utils.add_end_docstrings
|
||||||
|
|
||||||
|
[[autodoc]] utils.add_code_sample_docstrings
|
||||||
|
|
||||||
|
[[autodoc]] utils.replace_return_docstrings
|
||||||
|
|
||||||
|
## 特殊的属性
|
||||||
|
|
||||||
|
[[autodoc]] utils.cached_property
|
||||||
|
|
||||||
|
## 其他实用程序
|
||||||
|
|
||||||
|
[[autodoc]] utils._LazyModule
|
||||||
364
docs/source/zh/internal/generation_utils.md
Normal file
364
docs/source/zh/internal/generation_utils.md
Normal file
@@ -0,0 +1,364 @@
|
|||||||
|
<!--Copyright 2020 The HuggingFace Team. All rights reserved.
|
||||||
|
|
||||||
|
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
|
||||||
|
the License. You may obtain a copy of the License at
|
||||||
|
|
||||||
|
http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
|
||||||
|
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
|
||||||
|
an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
||||||
|
specific language governing permissions and limitations under the License.
|
||||||
|
|
||||||
|
⚠️ Note that this file is in Markdown but contain specific syntax for our doc-builder (similar to MDX) that may not be
|
||||||
|
rendered properly in your Markdown viewer.
|
||||||
|
|
||||||
|
-->
|
||||||
|
|
||||||
|
# 用于生成的工具
|
||||||
|
|
||||||
|
此页面列出了所有由 [`~generation.GenerationMixin.generate`],
|
||||||
|
[`~generation.GenerationMixin.greedy_search`],
|
||||||
|
[`~generation.GenerationMixin.contrastive_search`],
|
||||||
|
[`~generation.GenerationMixin.sample`],
|
||||||
|
[`~generation.GenerationMixin.beam_search`],
|
||||||
|
[`~generation.GenerationMixin.beam_sample`],
|
||||||
|
[`~generation.GenerationMixin.group_beam_search`], 和
|
||||||
|
[`~generation.GenerationMixin.constrained_beam_search`]使用的实用函数。
|
||||||
|
|
||||||
|
其中大多数仅在您研究库中生成方法的代码时才有用。
|
||||||
|
|
||||||
|
## 生成输出
|
||||||
|
|
||||||
|
[`~generation.GenerationMixin.generate`] 的输出是 [`~utils.ModelOutput`] 的一个子类的实例。这个输出是一种包含 [`~generation.GenerationMixin.generate`] 返回的所有信息数据结构，但也可以作为元组或字典使用。
|
||||||
|
这里是一个例子：
|
||||||
|
|
||||||
|
|
||||||
|
```python
|
||||||
|
from transformers import GPT2Tokenizer, GPT2LMHeadModel
|
||||||
|
|
||||||
|
tokenizer = GPT2Tokenizer.from_pretrained("gpt2")
|
||||||
|
model = GPT2LMHeadModel.from_pretrained("gpt2")
|
||||||
|
|
||||||
|
inputs = tokenizer("Hello, my dog is cute and ", return_tensors="pt")
|
||||||
|
generation_output = model.generate(**inputs, return_dict_in_generate=True, output_scores=True)
|
||||||
|
```
|
||||||
|
|
||||||
|
`generation_output` 的对象是 [`~generation.GreedySearchDecoderOnlyOutput`] 的一个实例，从该类的文档中我们可以看到，这意味着它具有以下属性：
|
||||||
|
|
||||||
|
- `sequences`: 生成的tokens序列
|
||||||
|
- `scores`（可选）: 每个生成步骤的语言建模头的预测分数
|
||||||
|
- `hidden_states`（可选）: 每个生成步骤模型的hidden states
|
||||||
|
- `attentions`（可选）: 每个生成步骤模型的注意力权重
|
||||||
|
|
||||||
|
在这里，由于我们传递了 `output_scores=True`，我们具有 `scores` 属性。但我们没有 `hidden_states` 和 `attentions`，因为没有传递 `output_hidden_states=True` 或 `output_attentions=True`。
|
||||||
|
|
||||||
|
您可以像通常一样访问每个属性，如果该属性未被模型返回，则将获得 `None`。例如，在这里 `generation_output.scores` 是语言建模头的所有生成预测分数，而 `generation_output.attentions` 为 `None`。
|
||||||
|
|
||||||
|
当我们将 `generation_output` 对象用作元组时，它只保留非 `None` 值的属性。例如，在这里它有两个元素，`loss` 然后是 `logits`，所以
|
||||||
|
|
||||||
|
|
||||||
|
```python
|
||||||
|
generation_output[:2]
|
||||||
|
```
|
||||||
|
|
||||||
|
将返回元组`(generation_output.sequences, generation_output.scores)`。
|
||||||
|
|
||||||
|
当我们将`generation_output`对象用作字典时，它只保留非`None`的属性。例如，它有两个键，分别是`sequences`和`scores`。
|
||||||
|
|
||||||
|
我们在此记录所有输出类型。
|
||||||
|
|
||||||
|
|
||||||
|
### PyTorch
|
||||||
|
|
||||||
|
[[autodoc]] generation.GreedySearchEncoderDecoderOutput
|
||||||
|
|
||||||
|
[[autodoc]] generation.GreedySearchDecoderOnlyOutput
|
||||||
|
|
||||||
|
[[autodoc]] generation.SampleEncoderDecoderOutput
|
||||||
|
|
||||||
|
[[autodoc]] generation.SampleDecoderOnlyOutput
|
||||||
|
|
||||||
|
[[autodoc]] generation.BeamSearchEncoderDecoderOutput
|
||||||
|
|
||||||
|
[[autodoc]] generation.BeamSearchDecoderOnlyOutput
|
||||||
|
|
||||||
|
[[autodoc]] generation.BeamSampleEncoderDecoderOutput
|
||||||
|
|
||||||
|
[[autodoc]] generation.BeamSampleDecoderOnlyOutput
|
||||||
|
|
||||||
|
[[autodoc]] generation.ContrastiveSearchEncoderDecoderOutput
|
||||||
|
|
||||||
|
[[autodoc]] generation.ContrastiveSearchDecoderOnlyOutput
|
||||||
|
|
||||||
|
### TensorFlow
|
||||||
|
|
||||||
|
[[autodoc]] generation.TFGreedySearchEncoderDecoderOutput
|
||||||
|
|
||||||
|
[[autodoc]] generation.TFGreedySearchDecoderOnlyOutput
|
||||||
|
|
||||||
|
[[autodoc]] generation.TFSampleEncoderDecoderOutput
|
||||||
|
|
||||||
|
[[autodoc]] generation.TFSampleDecoderOnlyOutput
|
||||||
|
|
||||||
|
[[autodoc]] generation.TFBeamSearchEncoderDecoderOutput
|
||||||
|
|
||||||
|
[[autodoc]] generation.TFBeamSearchDecoderOnlyOutput
|
||||||
|
|
||||||
|
[[autodoc]] generation.TFBeamSampleEncoderDecoderOutput
|
||||||
|
|
||||||
|
[[autodoc]] generation.TFBeamSampleDecoderOnlyOutput
|
||||||
|
|
||||||
|
[[autodoc]] generation.TFContrastiveSearchEncoderDecoderOutput
|
||||||
|
|
||||||
|
[[autodoc]] generation.TFContrastiveSearchDecoderOnlyOutput
|
||||||
|
|
||||||
|
### FLAX
|
||||||
|
|
||||||
|
[[autodoc]] generation.FlaxSampleOutput
|
||||||
|
|
||||||
|
[[autodoc]] generation.FlaxGreedySearchOutput
|
||||||
|
|
||||||
|
[[autodoc]] generation.FlaxBeamSearchOutput
|
||||||
|
|
||||||
|
## LogitsProcessor
|
||||||
|
|
||||||
|
[`LogitsProcessor`] 可以用于修改语言模型头的预测分数以进行生成
|
||||||
|
|
||||||
|
|
||||||
|
### PyTorch
|
||||||
|
|
||||||
|
[[autodoc]] AlternatingCodebooksLogitsProcessor
|
||||||
|
- __call__
|
||||||
|
|
||||||
|
[[autodoc]] ClassifierFreeGuidanceLogitsProcessor
|
||||||
|
- __call__
|
||||||
|
|
||||||
|
[[autodoc]] EncoderNoRepeatNGramLogitsProcessor
|
||||||
|
- __call__
|
||||||
|
|
||||||
|
[[autodoc]] EncoderRepetitionPenaltyLogitsProcessor
|
||||||
|
- __call__
|
||||||
|
|
||||||
|
[[autodoc]] EpsilonLogitsWarper
|
||||||
|
- __call__
|
||||||
|
|
||||||
|
[[autodoc]] EtaLogitsWarper
|
||||||
|
- __call__
|
||||||
|
|
||||||
|
[[autodoc]] ExponentialDecayLengthPenalty
|
||||||
|
- __call__
|
||||||
|
|
||||||
|
[[autodoc]] ForcedBOSTokenLogitsProcessor
|
||||||
|
- __call__
|
||||||
|
|
||||||
|
[[autodoc]] ForcedEOSTokenLogitsProcessor
|
||||||
|
- __call__
|
||||||
|
|
||||||
|
[[autodoc]] ForceTokensLogitsProcessor
|
||||||
|
- __call__
|
||||||
|
|
||||||
|
[[autodoc]] HammingDiversityLogitsProcessor
|
||||||
|
- __call__
|
||||||
|
|
||||||
|
[[autodoc]] InfNanRemoveLogitsProcessor
|
||||||
|
- __call__
|
||||||
|
|
||||||
|
[[autodoc]] LogitNormalization
|
||||||
|
- __call__
|
||||||
|
|
||||||
|
[[autodoc]] LogitsProcessor
|
||||||
|
- __call__
|
||||||
|
|
||||||
|
[[autodoc]] LogitsProcessorList
|
||||||
|
- __call__
|
||||||
|
|
||||||
|
[[autodoc]] LogitsWarper
|
||||||
|
- __call__
|
||||||
|
|
||||||
|
[[autodoc]] MinLengthLogitsProcessor
|
||||||
|
- __call__
|
||||||
|
|
||||||
|
[[autodoc]] MinNewTokensLengthLogitsProcessor
|
||||||
|
- __call__
|
||||||
|
|
||||||
|
[[autodoc]] NoBadWordsLogitsProcessor
|
||||||
|
- __call__
|
||||||
|
|
||||||
|
[[autodoc]] NoRepeatNGramLogitsProcessor
|
||||||
|
- __call__
|
||||||
|
|
||||||
|
[[autodoc]] PrefixConstrainedLogitsProcessor
|
||||||
|
- __call__
|
||||||
|
|
||||||
|
[[autodoc]] RepetitionPenaltyLogitsProcessor
|
||||||
|
- __call__
|
||||||
|
|
||||||
|
[[autodoc]] SequenceBiasLogitsProcessor
|
||||||
|
- __call__
|
||||||
|
|
||||||
|
[[autodoc]] SuppressTokensAtBeginLogitsProcessor
|
||||||
|
- __call__
|
||||||
|
|
||||||
|
[[autodoc]] SuppressTokensLogitsProcessor
|
||||||
|
- __call__
|
||||||
|
|
||||||
|
[[autodoc]] TemperatureLogitsWarper
|
||||||
|
- __call__
|
||||||
|
|
||||||
|
[[autodoc]] TopKLogitsWarper
|
||||||
|
- __call__
|
||||||
|
|
||||||
|
[[autodoc]] TopPLogitsWarper
|
||||||
|
- __call__
|
||||||
|
|
||||||
|
[[autodoc]] TypicalLogitsWarper
|
||||||
|
- __call__
|
||||||
|
|
||||||
|
[[autodoc]] UnbatchedClassifierFreeGuidanceLogitsProcessor
|
||||||
|
- __call__
|
||||||
|
|
||||||
|
[[autodoc]] WhisperTimeStampLogitsProcessor
|
||||||
|
- __call__
|
||||||
|
|
||||||
|
### TensorFlow
|
||||||
|
|
||||||
|
[[autodoc]] TFForcedBOSTokenLogitsProcessor
|
||||||
|
- __call__
|
||||||
|
|
||||||
|
[[autodoc]] TFForcedEOSTokenLogitsProcessor
|
||||||
|
- __call__
|
||||||
|
|
||||||
|
[[autodoc]] TFForceTokensLogitsProcessor
|
||||||
|
- __call__
|
||||||
|
|
||||||
|
[[autodoc]] TFLogitsProcessor
|
||||||
|
- __call__
|
||||||
|
|
||||||
|
[[autodoc]] TFLogitsProcessorList
|
||||||
|
- __call__
|
||||||
|
|
||||||
|
[[autodoc]] TFLogitsWarper
|
||||||
|
- __call__
|
||||||
|
|
||||||
|
[[autodoc]] TFMinLengthLogitsProcessor
|
||||||
|
- __call__
|
||||||
|
|
||||||
|
[[autodoc]] TFNoBadWordsLogitsProcessor
|
||||||
|
- __call__
|
||||||
|
|
||||||
|
[[autodoc]] TFNoRepeatNGramLogitsProcessor
|
||||||
|
- __call__
|
||||||
|
|
||||||
|
[[autodoc]] TFRepetitionPenaltyLogitsProcessor
|
||||||
|
- __call__
|
||||||
|
|
||||||
|
[[autodoc]] TFSuppressTokensAtBeginLogitsProcessor
|
||||||
|
- __call__
|
||||||
|
|
||||||
|
[[autodoc]] TFSuppressTokensLogitsProcessor
|
||||||
|
- __call__
|
||||||
|
|
||||||
|
[[autodoc]] TFTemperatureLogitsWarper
|
||||||
|
- __call__
|
||||||
|
|
||||||
|
[[autodoc]] TFTopKLogitsWarper
|
||||||
|
- __call__
|
||||||
|
|
||||||
|
[[autodoc]] TFTopPLogitsWarper
|
||||||
|
- __call__
|
||||||
|
|
||||||
|
### FLAX
|
||||||
|
|
||||||
|
[[autodoc]] FlaxForcedBOSTokenLogitsProcessor
|
||||||
|
- __call__
|
||||||
|
|
||||||
|
[[autodoc]] FlaxForcedEOSTokenLogitsProcessor
|
||||||
|
- __call__
|
||||||
|
|
||||||
|
[[autodoc]] FlaxForceTokensLogitsProcessor
|
||||||
|
- __call__
|
||||||
|
|
||||||
|
[[autodoc]] FlaxLogitsProcessor
|
||||||
|
- __call__
|
||||||
|
|
||||||
|
[[autodoc]] FlaxLogitsProcessorList
|
||||||
|
- __call__
|
||||||
|
|
||||||
|
[[autodoc]] FlaxLogitsWarper
|
||||||
|
- __call__
|
||||||
|
|
||||||
|
[[autodoc]] FlaxMinLengthLogitsProcessor
|
||||||
|
- __call__
|
||||||
|
|
||||||
|
[[autodoc]] FlaxSuppressTokensAtBeginLogitsProcessor
|
||||||
|
- __call__
|
||||||
|
|
||||||
|
[[autodoc]] FlaxSuppressTokensLogitsProcessor
|
||||||
|
- __call__
|
||||||
|
|
||||||
|
[[autodoc]] FlaxTemperatureLogitsWarper
|
||||||
|
- __call__
|
||||||
|
|
||||||
|
[[autodoc]] FlaxTopKLogitsWarper
|
||||||
|
- __call__
|
||||||
|
|
||||||
|
[[autodoc]] FlaxTopPLogitsWarper
|
||||||
|
- __call__
|
||||||
|
|
||||||
|
[[autodoc]] FlaxWhisperTimeStampLogitsProcessor
|
||||||
|
- __call__
|
||||||
|
|
||||||
|
## StoppingCriteria
|
||||||
|
|
||||||
|
可以使用[`StoppingCriteria`]来更改停止生成的时间（除了EOS token以外的方法）。请注意，这仅适用于我们的PyTorch实现。
|
||||||
|
|
||||||
|
|
||||||
|
[[autodoc]] StoppingCriteria
|
||||||
|
- __call__
|
||||||
|
|
||||||
|
[[autodoc]] StoppingCriteriaList
|
||||||
|
- __call__
|
||||||
|
|
||||||
|
[[autodoc]] MaxLengthCriteria
|
||||||
|
- __call__
|
||||||
|
|
||||||
|
[[autodoc]] MaxTimeCriteria
|
||||||
|
- __call__
|
||||||
|
|
||||||
|
## Constraints
|
||||||
|
|
||||||
|
可以使用[`Constraint`]来强制生成结果包含输出中的特定tokens或序列。请注意，这仅适用于我们的PyTorch实现。
|
||||||
|
|
||||||
|
[[autodoc]] Constraint
|
||||||
|
|
||||||
|
[[autodoc]] PhrasalConstraint
|
||||||
|
|
||||||
|
[[autodoc]] DisjunctiveConstraint
|
||||||
|
|
||||||
|
[[autodoc]] ConstraintListState
|
||||||
|
|
||||||
|
## BeamSearch
|
||||||
|
|
||||||
|
[[autodoc]] BeamScorer
|
||||||
|
- process
|
||||||
|
- finalize
|
||||||
|
|
||||||
|
[[autodoc]] BeamSearchScorer
|
||||||
|
- process
|
||||||
|
- finalize
|
||||||
|
|
||||||
|
[[autodoc]] ConstrainedBeamSearchScorer
|
||||||
|
- process
|
||||||
|
- finalize
|
||||||
|
|
||||||
|
## Utilities
|
||||||
|
|
||||||
|
[[autodoc]] top_k_top_p_filtering
|
||||||
|
|
||||||
|
[[autodoc]] tf_top_k_top_p_filtering
|
||||||
|
|
||||||
|
## Streamers
|
||||||
|
|
||||||
|
[[autodoc]] TextStreamer
|
||||||
|
|
||||||
|
[[autodoc]] TextIteratorStreamer
|
||||||
48
docs/source/zh/internal/image_processing_utils.md
Normal file
48
docs/source/zh/internal/image_processing_utils.md
Normal file
@@ -0,0 +1,48 @@
|
|||||||
|
<!--Copyright 2022 The HuggingFace Team. All rights reserved.
|
||||||
|
|
||||||
|
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
|
||||||
|
the License. You may obtain a copy of the License at
|
||||||
|
|
||||||
|
http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
|
||||||
|
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
|
||||||
|
an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
||||||
|
specific language governing permissions and limitations under the License.
|
||||||
|
|
||||||
|
⚠️ Note that this file is in Markdown but contain specific syntax for our doc-builder (similar to MDX) that may not be
|
||||||
|
rendered properly in your Markdown viewer.
|
||||||
|
|
||||||
|
-->
|
||||||
|
|
||||||
|
# Image Processors的工具
|
||||||
|
|
||||||
|
此页面列出了image processors使用的所有实用函数功能，主要是用于处理图像的功能变换。
|
||||||
|
|
||||||
|
其中大多数仅在您研究库中image processors的代码时有用。
|
||||||
|
|
||||||
|
|
||||||
|
## 图像转换
|
||||||
|
|
||||||
|
[[autodoc]] image_transforms.center_crop
|
||||||
|
|
||||||
|
[[autodoc]] image_transforms.center_to_corners_format
|
||||||
|
|
||||||
|
[[autodoc]] image_transforms.corners_to_center_format
|
||||||
|
|
||||||
|
[[autodoc]] image_transforms.id_to_rgb
|
||||||
|
|
||||||
|
[[autodoc]] image_transforms.normalize
|
||||||
|
|
||||||
|
[[autodoc]] image_transforms.pad
|
||||||
|
|
||||||
|
[[autodoc]] image_transforms.rgb_to_id
|
||||||
|
|
||||||
|
[[autodoc]] image_transforms.rescale
|
||||||
|
|
||||||
|
[[autodoc]] image_transforms.resize
|
||||||
|
|
||||||
|
[[autodoc]] image_transforms.to_pil_image
|
||||||
|
|
||||||
|
## ImageProcessingMixin
|
||||||
|
|
||||||
|
[[autodoc]] image_processing_utils.ImageProcessingMixin
|
||||||
83
docs/source/zh/internal/modeling_utils.md
Normal file
83
docs/source/zh/internal/modeling_utils.md
Normal file
@@ -0,0 +1,83 @@
|
|||||||
|
<!--Copyright 2020 The HuggingFace Team. All rights reserved.
|
||||||
|
|
||||||
|
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
|
||||||
|
the License. You may obtain a copy of the License at
|
||||||
|
|
||||||
|
http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
|
||||||
|
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
|
||||||
|
an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
||||||
|
specific language governing permissions and limitations under the License.
|
||||||
|
|
||||||
|
⚠️ Note that this file is in Markdown but contain specific syntax for our doc-builder (similar to MDX) that may not be
|
||||||
|
rendered properly in your Markdown viewer.
|
||||||
|
|
||||||
|
-->
|
||||||
|
|
||||||
|
# 自定义层和工具
|
||||||
|
|
||||||
|
此页面列出了库使用的所有自定义层，以及它为模型提供的实用函数。
|
||||||
|
|
||||||
|
其中大多数只有在您研究库中模型的代码时才有用。
|
||||||
|
|
||||||
|
|
||||||
|
## Pytorch自定义模块
|
||||||
|
|
||||||
|
[[autodoc]] pytorch_utils.Conv1D
|
||||||
|
|
||||||
|
[[autodoc]] modeling_utils.PoolerStartLogits
|
||||||
|
- forward
|
||||||
|
|
||||||
|
[[autodoc]] modeling_utils.PoolerEndLogits
|
||||||
|
- forward
|
||||||
|
|
||||||
|
[[autodoc]] modeling_utils.PoolerAnswerClass
|
||||||
|
- forward
|
||||||
|
|
||||||
|
[[autodoc]] modeling_utils.SquadHeadOutput
|
||||||
|
|
||||||
|
[[autodoc]] modeling_utils.SQuADHead
|
||||||
|
- forward
|
||||||
|
|
||||||
|
[[autodoc]] modeling_utils.SequenceSummary
|
||||||
|
- forward
|
||||||
|
|
||||||
|
## PyTorch帮助函数
|
||||||
|
|
||||||
|
[[autodoc]] pytorch_utils.apply_chunking_to_forward
|
||||||
|
|
||||||
|
[[autodoc]] pytorch_utils.find_pruneable_heads_and_indices
|
||||||
|
|
||||||
|
[[autodoc]] pytorch_utils.prune_layer
|
||||||
|
|
||||||
|
[[autodoc]] pytorch_utils.prune_conv1d_layer
|
||||||
|
|
||||||
|
[[autodoc]] pytorch_utils.prune_linear_layer
|
||||||
|
|
||||||
|
## TensorFlow自定义层
|
||||||
|
|
||||||
|
[[autodoc]] modeling_tf_utils.TFConv1D
|
||||||
|
|
||||||
|
[[autodoc]] modeling_tf_utils.TFSequenceSummary
|
||||||
|
|
||||||
|
## TensorFlow loss 函数
|
||||||
|
|
||||||
|
[[autodoc]] modeling_tf_utils.TFCausalLanguageModelingLoss
|
||||||
|
|
||||||
|
[[autodoc]] modeling_tf_utils.TFMaskedLanguageModelingLoss
|
||||||
|
|
||||||
|
[[autodoc]] modeling_tf_utils.TFMultipleChoiceLoss
|
||||||
|
|
||||||
|
[[autodoc]] modeling_tf_utils.TFQuestionAnsweringLoss
|
||||||
|
|
||||||
|
[[autodoc]] modeling_tf_utils.TFSequenceClassificationLoss
|
||||||
|
|
||||||
|
[[autodoc]] modeling_tf_utils.TFTokenClassificationLoss
|
||||||
|
|
||||||
|
## TensorFlow帮助函数
|
||||||
|
|
||||||
|
[[autodoc]] modeling_tf_utils.get_initializer
|
||||||
|
|
||||||
|
[[autodoc]] modeling_tf_utils.keras_serializable
|
||||||
|
|
||||||
|
[[autodoc]] modeling_tf_utils.shape_list
|
||||||
45
docs/source/zh/internal/pipelines_utils.md
Normal file
45
docs/source/zh/internal/pipelines_utils.md
Normal file
@@ -0,0 +1,45 @@
|
|||||||
|
<!--Copyright 2020 The HuggingFace Team. All rights reserved.
|
||||||
|
|
||||||
|
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
|
||||||
|
the License. You may obtain a copy of the License at
|
||||||
|
|
||||||
|
http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
|
||||||
|
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
|
||||||
|
an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
||||||
|
specific language governing permissions and limitations under the License.
|
||||||
|
|
||||||
|
⚠️ Note that this file is in Markdown but contain specific syntax for our doc-builder (similar to MDX) that may not be
|
||||||
|
rendered properly in your Markdown viewer.
|
||||||
|
|
||||||
|
-->
|
||||||
|
|
||||||
|
# pipelines的工具
|
||||||
|
|
||||||
|
|
||||||
|
此页面列出了库为pipelines提供的所有实用程序功能。
|
||||||
|
|
||||||
|
其中大多数只有在您研究库中模型的代码时才有用。
|
||||||
|
|
||||||
|
|
||||||
|
## 参数处理
|
||||||
|
|
||||||
|
[[autodoc]] pipelines.ArgumentHandler
|
||||||
|
|
||||||
|
[[autodoc]] pipelines.ZeroShotClassificationArgumentHandler
|
||||||
|
|
||||||
|
[[autodoc]] pipelines.QuestionAnsweringArgumentHandler
|
||||||
|
|
||||||
|
## 数据格式
|
||||||
|
|
||||||
|
[[autodoc]] pipelines.PipelineDataFormat
|
||||||
|
|
||||||
|
[[autodoc]] pipelines.CsvPipelineDataFormat
|
||||||
|
|
||||||
|
[[autodoc]] pipelines.JsonPipelineDataFormat
|
||||||
|
|
||||||
|
[[autodoc]] pipelines.PipedPipelineDataFormat
|
||||||
|
|
||||||
|
## 实用函数
|
||||||
|
|
||||||
|
[[autodoc]] pipelines.PipelineException
|
||||||
31
docs/source/zh/internal/time_series_utils.md
Normal file
31
docs/source/zh/internal/time_series_utils.md
Normal file
@@ -0,0 +1,31 @@
|
|||||||
|
<!--Copyright 2023 The HuggingFace Team. All rights reserved.
|
||||||
|
|
||||||
|
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
|
||||||
|
the License. You may obtain a copy of the License at
|
||||||
|
|
||||||
|
http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
|
||||||
|
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
|
||||||
|
an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
||||||
|
specific language governing permissions and limitations under the License.
|
||||||
|
|
||||||
|
⚠️ Note that this file is in Markdown but contain specific syntax for our doc-builder (similar to MDX) that may not be
|
||||||
|
rendered properly in your Markdown viewer.
|
||||||
|
|
||||||
|
-->
|
||||||
|
|
||||||
|
# 时间序列工具
|
||||||
|
|
||||||
|
|
||||||
|
此页面列出了可用于时间序列类模型的所有实用函数和类。
|
||||||
|
|
||||||
|
其中大多数仅在您研究时间序列模型的代码，或希望添加到分布输出类集合时有用。
|
||||||
|
|
||||||
|
|
||||||
|
## 输出分布
|
||||||
|
|
||||||
|
[[autodoc]] time_series_utils.NormalOutput
|
||||||
|
|
||||||
|
[[autodoc]] time_series_utils.StudentTOutput
|
||||||
|
|
||||||
|
[[autodoc]] time_series_utils.NegativeBinomialOutput
|
||||||
43
docs/source/zh/internal/tokenization_utils.md
Normal file
43
docs/source/zh/internal/tokenization_utils.md
Normal file
@@ -0,0 +1,43 @@
|
|||||||
|
<!--Copyright 2020 The HuggingFace Team. All rights reserved.
|
||||||
|
|
||||||
|
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
|
||||||
|
the License. You may obtain a copy of the License at
|
||||||
|
|
||||||
|
http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
|
||||||
|
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
|
||||||
|
an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
||||||
|
specific language governing permissions and limitations under the License.
|
||||||
|
|
||||||
|
⚠️ Note that this file is in Markdown but contain specific syntax for our doc-builder (similar to MDX) that may not be
|
||||||
|
rendered properly in your Markdown viewer.
|
||||||
|
|
||||||
|
-->
|
||||||
|
|
||||||
|
# Tokenizers的工具
|
||||||
|
|
||||||
|
并保留格式：此页面列出了tokenizers使用的所有实用函数，主要是类
|
||||||
|
[`~tokenization_utils_base.PreTrained TokenizerBase`] 实现了常用方法之间的
|
||||||
|
[`PreTrained Tokenizer`] 和 [`PreTrained TokenizerFast`] 以及混合类
|
||||||
|
[`~tokenization_utils_base.SpecialTokens Mixin`]。
|
||||||
|
|
||||||
|
其中大多数只有在您研究库中tokenizers的代码时才有用。
|
||||||
|
|
||||||
|
|
||||||
|
## PreTrainedTokenizerBase
|
||||||
|
|
||||||
|
[[autodoc]] tokenization_utils_base.PreTrainedTokenizerBase
|
||||||
|
- __call__
|
||||||
|
- all
|
||||||
|
|
||||||
|
## SpecialTokensMixin
|
||||||
|
|
||||||
|
[[autodoc]] tokenization_utils_base.SpecialTokensMixin
|
||||||
|
|
||||||
|
## Enums和namedtuples(命名元组)
|
||||||
|
|
||||||
|
[[autodoc]] tokenization_utils_base.TruncationStrategy
|
||||||
|
|
||||||
|
[[autodoc]] tokenization_utils_base.CharSpan
|
||||||
|
|
||||||
|
[[autodoc]] tokenization_utils_base.TokenSpan
|
||||||
50
docs/source/zh/internal/trainer_utils.md
Normal file
50
docs/source/zh/internal/trainer_utils.md
Normal file
@@ -0,0 +1,50 @@
|
|||||||
|
<!--Copyright 2020 The HuggingFace Team. All rights reserved.
|
||||||
|
|
||||||
|
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
|
||||||
|
the License. You may obtain a copy of the License at
|
||||||
|
|
||||||
|
http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
|
||||||
|
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
|
||||||
|
an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
||||||
|
specific language governing permissions and limitations under the License.
|
||||||
|
|
||||||
|
⚠️ Note that this file is in Markdown but contain specific syntax for our doc-builder (similar to MDX) that may not be
|
||||||
|
rendered properly in your Markdown viewer.
|
||||||
|
|
||||||
|
-->
|
||||||
|
|
||||||
|
# Trainer的工具
|
||||||
|
|
||||||
|
此页面列出了 [`Trainer`] 使用的所有实用函数。
|
||||||
|
|
||||||
|
其中大多数仅在您研究库中Trainer的代码时有用。
|
||||||
|
|
||||||
|
|
||||||
|
## 工具
|
||||||
|
|
||||||
|
[[autodoc]] EvalPrediction
|
||||||
|
|
||||||
|
[[autodoc]] IntervalStrategy
|
||||||
|
|
||||||
|
[[autodoc]] enable_full_determinism
|
||||||
|
|
||||||
|
[[autodoc]] set_seed
|
||||||
|
|
||||||
|
[[autodoc]] torch_distributed_zero_first
|
||||||
|
|
||||||
|
## Callbacks内部机制
|
||||||
|
|
||||||
|
[[autodoc]] trainer_callback.CallbackHandler
|
||||||
|
|
||||||
|
## 分布式评估
|
||||||
|
|
||||||
|
[[autodoc]] trainer_pt_utils.DistributedTensorGatherer
|
||||||
|
|
||||||
|
## Trainer参数解析
|
||||||
|
|
||||||
|
[[autodoc]] HfArgumentParser
|
||||||
|
|
||||||
|
## Debug工具
|
||||||
|
|
||||||
|
[[autodoc]] debug_utils.DebugUnderflowOverflow
|
||||||
Reference in New Issue
Block a user