Richard Lee be42c24d14 Honor trust_remote_code for custom tokenizers (#28854) ...

* pass through trust_remote_code for dynamically loading unregistered tokenizers specified by config
add test

* change directories back to previous directory after test

* fix ruff check

* Add a note to that block for future in case we want to remove it later

---------

Co-authored-by: Matt <rocketknight1@gmail.com>

2024-02-16 13:40:23 +00:00

..

__init__.py

Move test model folders (#17034)

2022-05-03 14:42:02 +02:00

test_configuration_auto.py

Update all references to canonical models (#29001)

2024-02-16 08:16:58 +01:00

test_feature_extraction_auto.py

Remote code improvements (#23959)

2023-06-06 14:31:14 -04:00

test_image_processing_auto.py

Remote code improvements (#23959)

2023-06-06 14:31:14 -04:00

test_modeling_auto.py

Fix slow backbone tests - out_indices must match stage name ordering (#28186)

2023-12-21 18:16:50 +00:00

test_modeling_flax_auto.py

Update all references to canonical models (#29001)

2024-02-16 08:16:58 +01:00

test_modeling_tf_auto.py

Update all references to canonical models (#29001)

2024-02-16 08:16:58 +01:00

test_modeling_tf_pytorch.py

Update all references to canonical models (#29001)

2024-02-16 08:16:58 +01:00

test_processor_auto.py

Don't save processor_config.json if a processor has no extra attribute (#28584)

2024-01-19 09:59:14 +00:00

test_tokenization_auto.py

Honor trust_remote_code for custom tokenizers (#28854)

2024-02-16 13:40:23 +00:00