Broken links fixed related to datasets docs (#27569)
fixed the broken links belogs to dataset library of transformers
This commit is contained in:
@@ -10,7 +10,7 @@ way which enables simple and efficient model parallelism.
|
||||
`run_image_captioning_flax.py` is a lightweight example of how to download and preprocess a dataset from the 🤗 Datasets
|
||||
library or use your own files (jsonlines or csv), then fine-tune one of the architectures above on it.
|
||||
|
||||
For custom datasets in `jsonlines` format please see: https://huggingface.co/docs/datasets/loading_datasets.html#json-files and you also will find examples of these below.
|
||||
For custom datasets in `jsonlines` format please see: https://huggingface.co/docs/datasets/loading_datasets#json-files and you also will find examples of these below.
|
||||
|
||||
### Download COCO dataset (2017)
|
||||
This example uses COCO dataset (2017) through a custom dataset script, which requires users to manually download the
|
||||
|
||||
@@ -494,7 +494,7 @@ def main():
|
||||
token=model_args.token,
|
||||
)
|
||||
# See more about loading any type of standard or custom dataset (from files, python dict, pandas DataFrame, etc) at
|
||||
# https://huggingface.co/docs/datasets/loading_datasets.html.
|
||||
# https://huggingface.co/docs/datasets/loading_datasets.
|
||||
|
||||
# Load pretrained model and tokenizer
|
||||
model = FlaxVisionEncoderDecoderModel.from_pretrained(
|
||||
|
||||
@@ -589,7 +589,7 @@ def main():
|
||||
num_proc=data_args.preprocessing_num_workers,
|
||||
)
|
||||
# See more about loading any type of standard or custom dataset (from files, python dict, pandas DataFrame, etc) at
|
||||
# https://huggingface.co/docs/datasets/loading_datasets.html.
|
||||
# https://huggingface.co/docs/datasets/loading_datasets.
|
||||
|
||||
# Load pretrained model and tokenizer
|
||||
|
||||
|
||||
@@ -484,7 +484,7 @@ def main():
|
||||
num_proc=data_args.preprocessing_num_workers,
|
||||
)
|
||||
# See more about loading any type of standard or custom dataset (from files, python dict, pandas DataFrame, etc) at
|
||||
# https://huggingface.co/docs/datasets/loading_datasets.html.
|
||||
# https://huggingface.co/docs/datasets/loading_datasets.
|
||||
|
||||
# Load pretrained model and tokenizer
|
||||
|
||||
|
||||
@@ -516,7 +516,7 @@ def main():
|
||||
num_proc=data_args.preprocessing_num_workers,
|
||||
)
|
||||
# See more about loading any type of standard or custom dataset (from files, python dict, pandas DataFrame, etc) at
|
||||
# https://huggingface.co/docs/datasets/loading_datasets.html.
|
||||
# https://huggingface.co/docs/datasets/loading_datasets.
|
||||
|
||||
# Load pretrained model and tokenizer
|
||||
|
||||
|
||||
@@ -630,7 +630,7 @@ def main():
|
||||
num_proc=data_args.preprocessing_num_workers,
|
||||
)
|
||||
# See more about loading any type of standard or custom dataset (from files, python dict, pandas DataFrame, etc) at
|
||||
# https://huggingface.co/docs/datasets/loading_datasets.html.
|
||||
# https://huggingface.co/docs/datasets/loading_datasets.
|
||||
|
||||
# Load pretrained model and tokenizer
|
||||
|
||||
|
||||
@@ -536,7 +536,7 @@ def main():
|
||||
token=model_args.token,
|
||||
)
|
||||
# See more about loading any type of standard or custom dataset (from files, python dict, pandas DataFrame, etc) at
|
||||
# https://huggingface.co/docs/datasets/loading_datasets.html.
|
||||
# https://huggingface.co/docs/datasets/loading_datasets.
|
||||
# endregion
|
||||
|
||||
# region Load pretrained model and tokenizer
|
||||
|
||||
@@ -9,7 +9,7 @@ way which enables simple and efficient model parallelism.
|
||||
|
||||
`run_summarization_flax.py` is a lightweight example of how to download and preprocess a dataset from the 🤗 Datasets library or use your own files (jsonlines or csv), then fine-tune one of the architectures above on it.
|
||||
|
||||
For custom datasets in `jsonlines` format please see: https://huggingface.co/docs/datasets/loading_datasets.html#json-files and you also will find examples of these below.
|
||||
For custom datasets in `jsonlines` format please see: https://huggingface.co/docs/datasets/loading_datasets#json-files and you also will find examples of these below.
|
||||
|
||||
### Train the model
|
||||
Next we can run the example script to train the model:
|
||||
|
||||
@@ -521,7 +521,7 @@ def main():
|
||||
token=model_args.token,
|
||||
)
|
||||
# See more about loading any type of standard or custom dataset (from files, python dict, pandas DataFrame, etc) at
|
||||
# https://huggingface.co/docs/datasets/loading_datasets.html.
|
||||
# https://huggingface.co/docs/datasets/loading_datasets.
|
||||
|
||||
# Load pretrained model and tokenizer
|
||||
|
||||
|
||||
@@ -410,7 +410,7 @@ def main():
|
||||
token=model_args.token,
|
||||
)
|
||||
# See more about loading any type of standard or custom dataset at
|
||||
# https://huggingface.co/docs/datasets/loading_datasets.html.
|
||||
# https://huggingface.co/docs/datasets/loading_datasets.
|
||||
|
||||
# Labels
|
||||
if data_args.task_name is not None:
|
||||
@@ -427,7 +427,7 @@ def main():
|
||||
num_labels = 1
|
||||
else:
|
||||
# A useful fast method:
|
||||
# https://huggingface.co/docs/datasets/package_reference/main_classes.html#datasets.Dataset.unique
|
||||
# https://huggingface.co/docs/datasets/package_reference/main_classes#datasets.Dataset.unique
|
||||
label_list = raw_datasets["train"].unique("label")
|
||||
label_list.sort() # Let's sort it for determinism
|
||||
num_labels = len(label_list)
|
||||
|
||||
@@ -465,7 +465,7 @@ def main():
|
||||
token=model_args.token,
|
||||
)
|
||||
# See more about loading any type of standard or custom dataset at
|
||||
# https://huggingface.co/docs/datasets/loading_datasets.html.
|
||||
# https://huggingface.co/docs/datasets/loading_datasets.
|
||||
|
||||
if raw_datasets["train"] is not None:
|
||||
column_names = raw_datasets["train"].column_names
|
||||
|
||||
Reference in New Issue
Block a user