Trainer automatically drops unused columns in nlp datasets (#6449)

* Add a classmethod to easily build a Trainer from nlp dataset and metric

* Fix docstrings

* Split train/eval

* Formatting

* Log dropped columns + docs

* Authorize callable activations

* Poc for auto activation

* Be framework-agnostic

* Formatting

* Remove class method

* Remove unnecessary code
This commit is contained in:
Sylvain Gugger
2020-08-20 16:29:14 -04:00
committed by GitHub
parent 5bf4465e6c
commit e5f452275b
3 changed files with 56 additions and 12 deletions

View File

@@ -64,6 +64,14 @@ except (ImportError, AssertionError):
_tf_available = False # pylint: disable=invalid-name
try:
import nlp # noqa: F401
_nlp_available = True
except ImportError:
_nlp_available = False
try:
from torch.hub import _get_torch_home
@@ -144,6 +152,10 @@ def is_torch_tpu_available():
return _torch_tpu_available
def is_nlp_available():
return _nlp_available
def is_psutil_available():
return _psutil_available