Add W&B backend for hyperparameter sweep (#14582)
# Add support for W&B hyperparameter sweep
This PR:
* allows using wandb for running hyperparameter search.
* The runs are visualized on W&B sweeps dashboard
* This supports runnning sweeps on parallel devices, all reporting to the same central dashboard.
### Usage
**To run new a hyperparameter search:**
```
trainer.hyperparameter_search(
backend="wandb",
project="transformers_sweep", # name of the project
n_trials=5,
metric="eval/loss", # metric to be optimized, default 'eval/loss'. A warning is raised if the passed metric is not found
)
```
This outputs a sweep id. Eg. `my_project/sweep_id`
**To run sweeps on parallel devices:**
Just pass sweep id which you want to run parallel
```
trainer.hyperparameter_search(
backend="wandb",
sweep_id = "my_project/sweep_id"
)
```
This commit is contained in:
@@ -59,7 +59,7 @@ from .file_utils import (
|
||||
is_torchaudio_available,
|
||||
is_vision_available,
|
||||
)
|
||||
from .integrations import is_optuna_available, is_ray_available, is_sigopt_available
|
||||
from .integrations import is_optuna_available, is_ray_available, is_sigopt_available, is_wandb_available
|
||||
|
||||
|
||||
SMALL_MODEL_IDENTIFIER = "julien-c/bert-xsmall-dummy"
|
||||
@@ -590,6 +590,19 @@ def require_sigopt(test_case):
|
||||
return test_case
|
||||
|
||||
|
||||
def require_wandb(test_case):
|
||||
"""
|
||||
Decorator marking a test that requires wandb.
|
||||
|
||||
These tests are skipped when wandb isn't installed.
|
||||
|
||||
"""
|
||||
if not is_wandb_available():
|
||||
return unittest.skip("test requires wandb")(test_case)
|
||||
else:
|
||||
return test_case
|
||||
|
||||
|
||||
def require_soundfile(test_case):
|
||||
"""
|
||||
Decorator marking a test that requires soundfile
|
||||
|
||||
Reference in New Issue
Block a user