[Wav2Vec2] PyCTCDecode Integration to support language model boosted decoding (#14339)

* up

* up

* up

* make it cleaner

* correct

* make styhahalal

* add more tests

* finish

* small fix

* make style

* up

* tryout to solve cicrle ci

* up

* fix more tests

* fix more tests

* apply sylvains suggestions

* fix import

* correct docs

* add pyctcdecode only to speech tests

* fix more tests

* add tf, flax and pt tests

* add pt

* fix last tests

* fix more tests

* Apply suggestions from code review

* change lines

* Apply suggestions from code review

Co-authored-by: Anton Lozhkov <aglozhkov@gmail.com>

* correct tests

* correct tests

* add doc string

Co-authored-by: Anton Lozhkov <aglozhkov@gmail.com>
This commit is contained in:
Patrick von Platen
2021-12-08 12:07:54 +01:00
committed by GitHub
parent 2e12d90b9e
commit 961732c276
16 changed files with 831 additions and 19 deletions

View File

@@ -36,8 +36,10 @@ from .file_utils import (
is_faiss_available,
is_flax_available,
is_keras2onnx_available,
is_librosa_available,
is_onnx_available,
is_pandas_available,
is_pyctcdecode_available,
is_pytesseract_available,
is_pytorch_quantization_available,
is_rjieba_available,
@@ -598,6 +600,26 @@ def require_deepspeed(test_case):
return test_case
def require_pyctcdecode(test_case):
"""
Decorator marking a test that requires pyctcdecode
"""
if not is_pyctcdecode_available():
return unittest.skip("test requires pyctcdecode")(test_case)
else:
return test_case
def require_librosa(test_case):
"""
Decorator marking a test that requires librosa
"""
if not is_librosa_available():
return unittest.skip("test requires librosa")(test_case)
else:
return test_case
def get_gpu_count():
"""
Return the number of available gpus (regardless of whether torch, tf or jax is used)