Big file_utils cleanup (#16396)

* Big file_utils cleanup

* This one still needs to be treated separately
This commit is contained in:
Sylvain Gugger
2022-03-25 07:25:20 -04:00
committed by GitHub
parent 2b23e0801a
commit 088c1880b7
222 changed files with 441 additions and 439 deletions

View File

@@ -381,7 +381,7 @@ important. Here is some advice is to make your debugging environment as efficien
original code so that you can directly input the ids instead of an input string.
- Make sure that the model in your debugging setup is **not** in training mode, which often causes the model to yield
random outputs due to multiple dropout layers in the model. Make sure that the forward pass in your debugging
environment is **deterministic** so that the dropout layers are not used. Or use *transformers.file_utils.set_seed*
environment is **deterministic** so that the dropout layers are not used. Or use *transformers.utils.set_seed*
if the old and new implementations are in the same framework.
The following section gives you more specific details/tips on how you can do this for *brand_new_bert*.

View File

@@ -12,35 +12,35 @@ specific language governing permissions and limitations under the License.
# General Utilities
This page lists all of Transformers general utility functions that are found in the file `file_utils.py`.
This page lists all of Transformers general utility functions that are found in the file `utils.py`.
Most of those are only useful if you are studying the general code in the library.
## Enums and namedtuples
[[autodoc]] file_utils.ExplicitEnum
[[autodoc]] utils.ExplicitEnum
[[autodoc]] file_utils.PaddingStrategy
[[autodoc]] utils.PaddingStrategy
[[autodoc]] file_utils.TensorType
[[autodoc]] utils.TensorType
## Special Decorators
[[autodoc]] file_utils.add_start_docstrings
[[autodoc]] utils.add_start_docstrings
[[autodoc]] file_utils.add_start_docstrings_to_model_forward
[[autodoc]] utils.add_start_docstrings_to_model_forward
[[autodoc]] file_utils.add_end_docstrings
[[autodoc]] utils.add_end_docstrings
[[autodoc]] file_utils.add_code_sample_docstrings
[[autodoc]] utils.add_code_sample_docstrings
[[autodoc]] file_utils.replace_return_docstrings
[[autodoc]] utils.replace_return_docstrings
## Special Properties
[[autodoc]] file_utils.cached_property
[[autodoc]] utils.cached_property
## Other Utilities
[[autodoc]] file_utils._LazyModule
[[autodoc]] utils._LazyModule

View File

@@ -25,7 +25,7 @@ Most of those are only useful if you are studying the code of the generate metho
## Generate Outputs
The output of [`~generation_utils.GenerationMixin.generate`] is an instance of a subclass of
[`~file_utils.ModelOutput`]. This output is a data structure containing all the information returned
[`~utils.ModelOutput`]. This output is a data structure containing all the information returned
by [`~generation_utils.GenerationMixin.generate`], but that can also be used as tuple or dictionary.
Here's an example:

View File

@@ -88,4 +88,4 @@ Due to Pytorch design, this functionality is only available for floating dtypes.
## Pushing to the Hub
[[autodoc]] file_utils.PushToHubMixin
[[autodoc]] utils.PushToHubMixin

View File

@@ -12,7 +12,7 @@ specific language governing permissions and limitations under the License.
# Model outputs
All models have outputs that are instances of subclasses of [`~file_utils.ModelOutput`]. Those are
All models have outputs that are instances of subclasses of [`~utils.ModelOutput`]. Those are
data structures containing all the information returned by the model, but that can also be used as tuples or
dictionaries.
@@ -57,7 +57,7 @@ documented on their corresponding model page.
## ModelOutput
[[autodoc]] file_utils.ModelOutput
[[autodoc]] utils.ModelOutput
- to_tuple
## BaseModelOutput

View File

@@ -40,7 +40,7 @@ The [`Trainer`] contains the basic training loop which supports the above featur
The [`Trainer`] class is optimized for 🤗 Transformers models and can have surprising behaviors
when you use it on other models. When using it on your own model, make sure:
- your model always return tuples or subclasses of [`~file_utils.ModelOutput`].
- your model always return tuples or subclasses of [`~utils.ModelOutput`].
- your model can compute the loss if a `labels` argument is provided and that loss is returned as the first
element of the tuple (if your model returns tuples)
- your model can accept multiple label arguments (use the `label_names` in your [`TrainingArguments`] to indicate their name to the [`Trainer`]) but none of them should be named `"label"`.

View File

@@ -855,7 +855,7 @@ If you need to switch a tensor to bf16, it's just: `t.to(dtype=torch.bfloat16)`
Here is how you can check if your setup supports bf16:
```
python -c 'import transformers; print(f"BF16 support is {transformers.file_utils.is_torch_bf16_available()}")'
python -c 'import transformers; print(f"BF16 support is {transformers.utils.is_torch_bf16_available()}")'
```
On the other hand bf16 has a much worse precision than fp16, so there are certain situations where you'd still want to use fp16 and not bf16.