SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity

Auto processor fix (#14623)

Browse Source 
* Add AutoProcessor class
Init and tests
Add doc
Fix init
Update src/transformers/models/auto/processing_auto.py

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
Reverts to tokenizer or feature extractor when available
Adapt test

* Revert "Adapt test"

This reverts commit bbdde5fab02465f24b54b227390073082cb32093.

* Revert "Reverts to tokenizer or feature extractor when available"

This reverts commit 77659ff5d21b6cc0baf6f443017e35e056a525bb.

* Don't revert everything Lysandre!

Co-authored-by: Sylvain Gugger <sylvain.gugger@gmail.com>
This commit is contained in:
Lysandre Debut
2021-12-06 18:49:50 +01:00
committed by GitHub
parent cbe6026536
commit e9688875bf
2 changed files with 3 additions and 31 deletions
Download Patch File Download Diff File Expand all files Collapse all files

12
tests/test_processor_auto.py
View File

@@ -17,8 +17,7 @@ import os
import tempfile
import unittest
from transformers import AutoProcessor, BeitFeatureExtractor, BertTokenizerFast, Wav2Vec2Config, Wav2Vec2Processor
from transformers.testing_utils import require_torch
from transformers import AutoProcessor, Wav2Vec2Config, Wav2Vec2Processor
SAMPLE_PROCESSOR_CONFIG_DIR = os.path.join(os.path.dirname(os.path.abspath(__file__)), "fixtures")
@@ -45,12 +44,3 @@ class AutoFeatureExtractorTest(unittest.TestCase):
processor = AutoProcessor.from_pretrained(tmpdirname)
self.assertIsInstance(processor, Wav2Vec2Processor)
def test_auto_processor_reverts_to_tokenizer(self):
processor = AutoProcessor.from_pretrained("bert-base-cased")
self.assertIsInstance(processor, BertTokenizerFast)
@require_torch
def test_auto_processor_reverts_to_feature_extractor(self):
processor = AutoProcessor.from_pretrained("microsoft/beit-base-patch16-224")
self.assertIsInstance(processor, BeitFeatureExtractor)