SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity

[RAG] Add Ray implementation for distributed retrieval (#9197)

Browse Source 
* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* uncomment

* uncomment

* wip

* updates

* add docstring

* updates

* fix arg

* fixes

* add unit tests

* update readme

* update readme

* update finetune script

* update test

* add test

* add ray to test dependencies

* separate ray and ray tune

* formatting

* shutdown ray at end of test

* fix tests

* formatting

* formatting

* even more formatting

* address comments

* formatting

* add files

* Update examples/research_projects/rag/test_distributed_retriever.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* address comments

* addressing comments

Co-authored-by: Ubuntu <ubuntu@ip-172-31-21-208.us-west-2.compute.internal>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
This commit is contained in:
Amog Kamsetty
2020-12-21 01:39:30 -08:00
committed by GitHub
parent f38c4ad302
commit a4b21cdd20
14 changed files with 561 additions and 56 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
src/transformers/trainer.py
View File

@@ -37,7 +37,7 @@ from .integrations import ( # isort: split
is_fairscale_available,
is_mlflow_available,
is_optuna_available,
is_ray_available,
is_ray_tune_available,
is_tensorboard_available,
is_wandb_available,
run_hp_search_optuna,
@@ -145,7 +145,7 @@ if is_mlflow_available():
if is_optuna_available():
import optuna
if is_ray_available():
if is_ray_tune_available():
from ray import tune
if is_azureml_available():
@@ -1062,7 +1062,7 @@ class Trainer:
backend = HPSearchBackend(backend)
if backend == HPSearchBackend.OPTUNA and not is_optuna_available():
raise RuntimeError("You picked the optuna backend, but it is not installed. Use `pip install optuna`.")
if backend == HPSearchBackend.RAY and not is_ray_available():
if backend == HPSearchBackend.RAY and not is_ray_tune_available():
raise RuntimeError(
"You picked the Ray Tune backend, but it is not installed. Use `pip install 'ray[tune]'`."
)