Set cache_dir for evaluate.load() in example scripts (#28422)
While using `run_clm.py`,[^1] I noticed that some files were being added
to my global cache, not the local cache. I set the `cache_dir` parameter
for the one call to `evaluate.load()`, which partially solved the
problem. I figured that while I was fixing the one script upstream, I
might as well fix the problem in all other example scripts that I could.
There are still some files being added to my global cache, but this
appears to be a bug in `evaluate` itself. This commit at least moves
some of the files into the local cache, which is better than before.
To create this PR, I made the following regex-based transformation:
`evaluate\.load\((.*?)\)` -> `evaluate\.load\($1,
cache_dir=model_args.cache_dir\)`. After using that, I manually fixed
all modified files with `ruff` serving as useful guidance. During the
process, I removed one existing usage of the `cache_dir` parameter in a
script that did not have a corresponding `--cache-dir` argument
declared.
[^1]: I specifically used `pytorch/language-modeling/run_clm.py` from
v4.34.1 of the library. For the original code, see the following URL:
acc394c4f5/examples/pytorch/language-modeling/run_clm.py.
This commit is contained in:
@@ -680,7 +680,7 @@ def main():
|
||||
# instantiate a data collator and the trainer
|
||||
|
||||
# Define evaluation metrics during training, *i.e.* word error rate, character error rate
|
||||
eval_metrics = {metric: evaluate.load(metric) for metric in data_args.eval_metrics}
|
||||
eval_metrics = {metric: evaluate.load(metric, cache_dir=model_args.cache_dir) for metric in data_args.eval_metrics}
|
||||
|
||||
# for large datasets it is advised to run the preprocessing on a
|
||||
# single machine first with ``args.preprocessing_only`` since there will mostly likely
|
||||
|
||||
@@ -702,7 +702,7 @@ def main():
|
||||
# instantiate a data collator and the trainer
|
||||
|
||||
# Define evaluation metrics during training, *i.e.* word error rate, character error rate
|
||||
eval_metrics = {metric: evaluate.load(metric) for metric in data_args.eval_metrics}
|
||||
eval_metrics = {metric: evaluate.load(metric, cache_dir=model_args.cache_dir) for metric in data_args.eval_metrics}
|
||||
|
||||
# for large datasets it is advised to run the preprocessing on a
|
||||
# single machine first with ``args.preprocessing_only`` since there will mostly likely
|
||||
|
||||
@@ -520,7 +520,7 @@ def main():
|
||||
return
|
||||
|
||||
# 8. Load Metric
|
||||
metric = evaluate.load("wer")
|
||||
metric = evaluate.load("wer", cache_dir=model_args.cache_dir)
|
||||
|
||||
def compute_metrics(pred):
|
||||
pred_ids = pred.predictions
|
||||
|
||||
Reference in New Issue
Block a user