Emin Orhan
d47a9e8ce5
fix redundant checkpointing in example training scripts ( #33131 )
...
* fix redundant checkpointing in example scripts
* Update examples/pytorch/image-classification/run_image_classification_no_trainer.py
Co-authored-by: Marc Sun <57196510+SunMarc@users.noreply.github.com >
* Update examples/pytorch/translation/run_translation_no_trainer.py
Co-authored-by: Marc Sun <57196510+SunMarc@users.noreply.github.com >
* Update examples/pytorch/token-classification/run_ner_no_trainer.py
Co-authored-by: Marc Sun <57196510+SunMarc@users.noreply.github.com >
* Update examples/pytorch/text-classification/run_glue_no_trainer.py
Co-authored-by: Marc Sun <57196510+SunMarc@users.noreply.github.com >
* Update examples/pytorch/summarization/run_summarization_no_trainer.py
Co-authored-by: Marc Sun <57196510+SunMarc@users.noreply.github.com >
* Update examples/pytorch/semantic-segmentation/run_semantic_segmentation_no_trainer.py
Co-authored-by: Marc Sun <57196510+SunMarc@users.noreply.github.com >
* Update examples/pytorch/language-modeling/run_mlm_no_trainer.py
Co-authored-by: Marc Sun <57196510+SunMarc@users.noreply.github.com >
* Update examples/pytorch/language-modeling/run_fim_no_trainer.py
Co-authored-by: Marc Sun <57196510+SunMarc@users.noreply.github.com >
* Update examples/pytorch/language-modeling/run_clm_no_trainer.py
Co-authored-by: Marc Sun <57196510+SunMarc@users.noreply.github.com >
* Update examples/pytorch/image-pretraining/run_mim_no_trainer.py
Co-authored-by: Marc Sun <57196510+SunMarc@users.noreply.github.com >
* Update examples/pytorch/instance-segmentation/run_instance_segmentation_no_trainer.py
Co-authored-by: Marc Sun <57196510+SunMarc@users.noreply.github.com >
* Update examples/pytorch/multiple-choice/run_swag_no_trainer.py
Co-authored-by: Marc Sun <57196510+SunMarc@users.noreply.github.com >
* Update examples/pytorch/question-answering/run_qa_no_trainer.py
Co-authored-by: Marc Sun <57196510+SunMarc@users.noreply.github.com >
* Update examples/pytorch/object-detection/run_object_detection_no_trainer.py
Co-authored-by: Marc Sun <57196510+SunMarc@users.noreply.github.com >
* Update examples/pytorch/question-answering/run_qa_beam_search_no_trainer.py
Co-authored-by: Marc Sun <57196510+SunMarc@users.noreply.github.com >
---------
Co-authored-by: Marc Sun <57196510+SunMarc@users.noreply.github.com >
2024-08-27 15:50:00 +02:00
Arthur Zucker
26a9443dae
dev version 4.45.0
2024-08-06 18:33:18 +02:00
Lysandre
ff0d708fe6
Dev version: v4.44.0.dev0
2024-07-23 17:12:47 +02:00
Lysandre
727eea4ab0
v4.43.0.dev0
2024-06-27 17:40:07 +02:00
Albert Villanova del Moral
a14b055b65
Pass datasets trust_remote_code ( #31406 )
...
* Pass datasets trust_remote_code
* Pass trust_remote_code in more tests
* Add trust_remote_dataset_code arg to some tests
* Revert "Temporarily pin datasets upper version to fix CI"
This reverts commit b7672826ca833fc17a3e
2024-06-17 17:29:13 +01:00
Pavel Iakubovskii
517df566f5
Decorators for deprecation and named arguments validation ( #30799 )
...
* Fix do_reduce_labels for maskformer image processor
* Deprecate reduce_labels in favor to do_reduce_labels
* Deprecate reduce_labels in favor to do_reduce_labels (segformer)
* Deprecate reduce_labels in favor to do_reduce_labels (oneformer)
* Deprecate reduce_labels in favor to do_reduce_labels (maskformer)
* Deprecate reduce_labels in favor to do_reduce_labels (mask2former)
* Fix typo
* Update mask2former test
* fixup
* Update segmentation examples
* Update docs
* Fixup
* Imports fixup
* Add deprecation decorator draft
* Add deprecation decorator
* Fixup
* Add deprecate_kwarg decorator
* Validate kwargs decorator
* Kwargs validation (beit)
* fixup
* Kwargs validation (mask2former)
* Kwargs validation (maskformer)
* Kwargs validation (oneformer)
* Kwargs validation (segformer)
* Better message
* Fix oneformer processor save-load test
* Update src/transformers/utils/deprecation.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com >
* Update src/transformers/utils/deprecation.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com >
* Update src/transformers/utils/deprecation.py
Co-authored-by: Pablo Montalvo <39954772+molbap@users.noreply.github.com >
* Update src/transformers/utils/deprecation.py
Co-authored-by: Pablo Montalvo <39954772+molbap@users.noreply.github.com >
* Better handle classmethod warning
* Fix typo, remove warn
* Add header
* Docs and `additional_message`
* Move to filter decorator ot generic
* Proper deprecation for semantic segm scripts
* Add to __init__ and update import
* Basic tests for filter decorator
* Fix doc
* Override `to_dict()` to pop depracated `_max_size`
* Pop unused parameters
* Fix trailing whitespace
* Add test for deprecation
* Add deprecation warning control parameter
* Update generic test
* Fixup deprecation tests
* Introduce init service kwargs
* Revert popping unused params
* Revert oneformer test
* Allow "metadata" to pass
* Better docs
* Fix test
* Add notion in docstring
* Fix notification for both names
* Add func name to warning message
* Fixup
---------
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com >
Co-authored-by: Pablo Montalvo <39954772+molbap@users.noreply.github.com >
2024-06-10 12:35:10 +01:00
Arthur
673440d073
update ruff version ( #30932 )
...
* update ruff version
* fix research projects
* Empty
* Fix errors
---------
Co-authored-by: Lysandre <lysandre@huggingface.co >
2024-05-22 06:40:15 +02:00
Arthur Zucker
b6eb708bf1
v4.42.dev.0
2024-05-17 17:30:41 +02:00
amyeroberts
57c965a8f1
Remove deprecated logic and warnings ( #30743 )
...
* Remove deprecated logic and warnings
* Add back some code that seems to be important...
* Let's just add all he nllb stuff back; removing it is a bit more involved
* Remove kwargs
* Remove more kwargs
2024-05-17 12:15:59 +01:00
Zach Mueller
60d5f8f9f0
🚨 🚨 🚨 Deprecate evaluation_strategy to eval_strategy 🚨 🚨 🚨 ( #30190 )
...
* Alias
* Note alias
* Tests and src
* Rest
* Clean
* Change typing?
* Fix tests
* Deprecation versions
2024-04-18 12:49:43 -04:00
Lysandre
ce8e64fbe2
Dev version
2024-04-18 15:53:25 +02:00
Pavel Iakubovskii
56d001b26f
Fix and simplify semantic-segmentation example ( #30145 )
...
* Remove unused augmentation
* Fix pad_if_smaller() and remove unused augmentation
* Add indentation
* Fix requirements
* Update dataset use instructions
* Replace transforms with albumentations
* Replace identity transform with None
* Fixing formatting
* Fixed comment place
2024-04-10 09:10:52 +01:00
NielsRogge
e9c23fa056
[Trainer] Undo #29896 ( #30129 )
...
* Undo
* Use tokenizer
* Undo data collator
2024-04-09 12:55:42 +02:00
NielsRogge
1ab7136488
[Trainer] Allow passing image processor ( #29896 )
...
* Add image processor to trainer
* Replace tokenizer=image_processor everywhere
2024-04-05 10:10:44 +02:00
Arthur Zucker
1248f09252
v4.40.0.dev.0
2024-03-20 23:31:47 +09:00
Hilco van der Wilk
b6404866cd
Update legacy Repository usage in various example files ( #29085 )
...
* Update legacy Repository usage in `examples/pytorch/text-classification/run_glue_no_trainer.py`
Marked for deprecation here https://huggingface.co/docs/huggingface_hub/guides/upload#legacy-upload-files-with-git-lfs
* Fix import order
* Replace all example usage of deprecated Repository
* Fix remaining repo call and rename args variable
* Revert removing creation of gitignore files and don't change research examples
2024-03-12 13:20:49 +00:00
Arthur Zucker
1a77f07f65
v4.39.dev.0
2024-02-21 15:23:22 +09:00
Klaus Hipp
721ee783ca
[Docs] Fix spelling and grammar mistakes ( #28825 )
...
* Fix typos and grammar mistakes in docs and examples
* Fix typos in docstrings and comments
* Fix spelling of `tokenizer` in model tests
* Remove erroneous spaces in decorators
* Remove extra spaces in Markdown link texts
2024-02-02 08:45:00 +01:00
zspo
d98591a12b
[docs] fix some bugs about parameter description ( #28806 )
...
Co-authored-by: p_spozzhang <p_spozzhang@tencent.com >
2024-02-01 16:59:29 +00:00
Steven Liu
abe0289e6d
[docs] Fix datasets in guides ( #28715 )
...
* change datasets
* fix
2024-01-26 09:29:07 -08:00
bofeng huang
deb2b59073
Fix lr_scheduler in no_trainer training scripts ( #27872 )
...
* Fix lr_scheduler
* Fix lr scheduler
2024-01-22 14:22:18 +00:00
Amy Roberts
b2748a6efd
v4.38.dev.0
2024-01-19 10:43:28 +00:00
Alex Hedges
95091e1582
Set cache_dir for evaluate.load() in example scripts ( #28422 )
...
While using `run_clm.py`,[^1] I noticed that some files were being added
to my global cache, not the local cache. I set the `cache_dir` parameter
for the one call to `evaluate.load()`, which partially solved the
problem. I figured that while I was fixing the one script upstream, I
might as well fix the problem in all other example scripts that I could.
There are still some files being added to my global cache, but this
appears to be a bug in `evaluate` itself. This commit at least moves
some of the files into the local cache, which is better than before.
To create this PR, I made the following regex-based transformation:
`evaluate\.load\((.*?)\)` -> `evaluate\.load\($1,
cache_dir=model_args.cache_dir\)`. After using that, I manually fixed
all modified files with `ruff` serving as useful guidance. During the
process, I removed one existing usage of the `cache_dir` parameter in a
script that did not have a corresponding `--cache-dir` argument
declared.
[^1]: I specifically used `pytorch/language-modeling/run_clm.py` from
v4.34.1 of the library. For the original code, see the following URL:
acc394c4f5/examples/pytorch/language-modeling/run_clm.py
2024-01-11 15:38:44 +01:00
Lysandre
3ed3e3190c
Dev version
2023-12-13 18:29:31 +01:00
Lysandre
bc78fd1274
Dev version
2023-11-02 18:15:36 +01:00
Dong-geon Lee
25e6e9418c
Unify warning styles for better readability ( #27184 )
2023-10-31 18:12:14 +00:00
Lucain
66b088faf0
Provide alternative when warning on use_auth_token ( #27105 )
2023-10-27 14:32:54 +02:00
Tom Aarsen
40ea9ab2a1
Add many missing spaces in adjacent strings ( #26751 )
...
Add missing spaces in adjacent strings
2023-10-12 10:28:40 +02:00
Zach Mueller
1d6a84749b
Fix checkpoint path in no_trainer scripts ( #26733 )
...
checkpoint path
2023-10-11 16:16:27 +02:00
Lysandre
bd6205919a
v4.35.0.dev0
2023-10-03 16:54:37 +02:00
Phuc Van Phan
4fb64e285a
chore: correct update_step and correct gradient_accumulation_steps ( #26068 )
2023-09-12 18:31:23 +01:00
Phuc Van Phan
5af2c62696
docs: add space to docs ( #26067 )
...
* docs: add space to docs
* docs: remove reduntant space
2023-09-11 22:03:26 +01:00
Lysandre
d8e13b3e04
v4.34.dev.0
2023-09-04 15:12:11 -04:00
Sylvain Gugger
5c67682b16
v4.33.0.dev0
2023-08-21 07:07:04 -04:00
Zach Mueller
01ab39b65f
Load state in else ( #25318 )
...
* Load else
* New approach
* Propagate
2023-08-08 05:41:00 -04:00
Jackmin801
145109382a
Allow trust_remote_code in example scripts ( #25248 )
...
* pytorch examples
* pytorch mim no trainer
* cookiecutter
* flax examples
* missed line in pytorch run_glue
* tensorflow examples
* tensorflow run_clip
* tensorflow run_mlm
* tensorflow run_ner
* tensorflow run_clm
* pytorch example from_configs
* pytorch no trainer examples
* Revert "tensorflow run_clip"
This reverts commit 261f86ac1f1c9e05dd3fd0291e1a1f8e573781d5.
* fix: duplicated argument
2023-08-07 16:32:25 +02:00
Yih-Dar
149cb0cce2
Add token arugment in example scripts ( #25172 )
...
* fix
* fix
* fix
* fix
* fix
* fix
* fix
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com >
2023-08-02 11:17:31 +02:00
Yih-Dar
d53b8ad780
Update use_auth_token -> token in example scripts ( #25167 )
...
* pytorch examples
* tensorflow examples
* flax examples
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com >
2023-07-28 15:33:45 +02:00
Lucain
6232c380f2
Fix .push_to_hub and cleanup get_full_repo_name usage ( #25120 )
...
* Fix .push_to_hub and cleanup get_full_repo_name usage
* Do not rely on Python bool conversion magic
* request changes
2023-07-28 11:40:08 +02:00
Zach Mueller
aa1b09c5d1
Change logic for logging in the examples ( #24956 )
...
Change logic
2023-07-20 12:30:10 -04:00
Sylvain Gugger
e9ad51306f
4.32.0.dev0
2023-07-17 13:30:44 -04:00
Ethan
f7d80cb3d2
Fix steps bugs in no trainer examples ( #24197 )
...
Fix step bugs in no trainer + load checkpoint + grad acc
2023-06-12 11:49:55 -04:00
Sylvain Gugger
ba695c1efd
v4.31.0.dev0
2023-06-07 16:49:00 -04:00
Zachary Mueller
072188d638
Act on deprecations in Accelerate no_trainer examples ( #24053 )
...
Act on deprecation
2023-06-06 13:04:38 -04:00
Zachary Mueller
b191d7db44
Update all no_trainer with skip_first_batches ( #23664 )
2023-05-22 14:49:31 -04:00
Hari
5d02e6bd20
Convert numpy arrays to lists before saving the evaluation metrics as json ( #23268 )
...
* convert numpy array to list before writing to json
per_category_iou and per_category_accuracy are ndarray in the eval_metrics
* code reformatted with make style
2023-05-11 08:54:23 -04:00
Sylvain Gugger
a0c0a78233
v4.30.0.dev0
2023-05-09 14:59:38 -04:00
Sylvain Gugger
888c4a2ae0
v4.29.0.dev0
2023-04-12 20:04:29 -04:00
Sylvain Gugger
ebdb185bef
v4.28.0.dev0
2023-03-14 13:49:10 -04:00
Sylvain Gugger
b19d64d852
Respect documentation on passive log level ( #21700 )
...
* Respect documentation on passive log level
* Fix test and set log level in examples
* Add doc
2023-02-22 09:39:18 +01:00
