SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity
Files
f3d2f7a6e08efe18debf59512325f02128394b43
HuggingFace_transformer/tests/models
History
NielsRogge f3d2f7a6e0 Add MarkupLM (#19198) 
* First draft

* Make basic test work

* Fix most tokenizer tests

* More improvements

* Make more tests pass

* Fix more tests

* Fix some code quality

* Improve truncation

* Implement feature extractor

* Improve feature extractor and add tests

* Improve feature extractor tests

* Fix pair_input test partly

* Add fast tokenizer

* Improve implementation

* Fix rebase

* Fix rebase

* Fix most of the tokenizer tests.

* propose solution for fast

* add: integration test for fasttokenizer, warning for decode, fix template in slow tokenizer

* add: modify markuplmconverter

* add: some modify on converter and tokenizerfast

* Fix style, copies

* Make fixup

* Update tokenization_markuplm.py

* Update test_tokenization_markuplm.py

* Update markuplm related

* Improve processor, add integration test

* Add processor test file

* Improve processor

* Improve processor tests

* Fix more processor tests

* Fix processor tests

* Update docstrings

* Add Copied from statements

* Add more Copied from statements

* Add code examples

* Improve code examples

* Add model to doc tests

* Adding dependency check

* Add dummy file

* Add requires_backends

* Add model to toctree

* Fix more things, disable dependency check for now

* Apply more suggestions

* Add soft dependency

* Add annotators to tests

* Fix style

* Remove from_slow=True

* Remove print statements

* Add sanity check

* Fix processor test

* Fix processor tests, add more docs

* Add doc tests for mdx file

* Add more tips

* Apply suggestions

Co-authored-by: Niels Rogge <nielsrogge@Nielss-MacBook-Pro.local>
Co-authored-by: lockon-n <45759388+lockon-n@users.noreply.github.com>
Co-authored-by: SaulLu <lucilesaul.com@gmail.com>
Co-authored-by: lockon-n <dd098309@126.com>
2022-09-30 08:25:43 +02:00
..
albert
…
auto
Make AutoProcessor a magic loading class for all modalities (#18963)
2022-09-14 07:36:12 -04:00
bart
TF: TF 2.10 unpin + related onnx test skips (#18995)
2022-09-12 19:30:27 +01:00
barthez
…
bartpho
…
beit
Fix BeitFeatureExtractor postprocessing (#19119)
2022-09-20 18:53:40 +03:00
bert
TF: TF 2.10 unpin + related onnx test skips (#18995)
2022-09-12 19:30:27 +01:00
bert_generation
Black preview (#17217)
2022-05-12 16:25:55 -04:00
bert_japanese
Black preview (#17217)
2022-05-12 16:25:55 -04:00
bertweet
…
big_bird
Use higher value for hidden_size in Flax BigBird test (#17822)
2022-06-24 19:31:30 +02:00
bigbird_pegasus
Black preview (#17217)
2022-05-12 16:25:55 -04:00
blenderbot
Update serving code to enable saved_model=True (#18153)
2022-07-22 18:05:38 +01:00
blenderbot_small
Update serving code to enable saved_model=True (#18153)
2022-07-22 18:05:38 +01:00
bloom
Use assertAlmostEqual in BloomEmbeddingTest.test_logits (#19200)
2022-09-26 14:56:41 +02:00
bort
…
byt5
Replace as_target context managers by direct calls (#18325)
2022-07-29 08:09:09 -04:00
camembert
…
canine
Replace as_target context managers by direct calls (#18325)
2022-07-29 08:09:09 -04:00
clip
Update serving code to enable saved_model=True (#18153)
2022-07-22 18:05:38 +01:00
codegen
Update expected values in CodeGen tests (#17888)
2022-07-01 15:33:36 +02:00
conditional_detr
Add support for conditional detr (#18948)
2022-09-22 09:45:04 +02:00
convbert
…
convnext
FX support for ConvNext, Wav2Vec2 and ResNet (#19053)
2022-09-16 10:57:41 +02:00
cpm
…
ctrl
Fix CTRL tests (#17508)
2022-06-01 16:27:23 +02:00
cvt
has_attentions - consistent test skipping logic and tf tests (#17495)
2022-06-09 09:50:03 +02:00
data2vec
skip some test_multi_gpu_data_parallel_forward (#18188)
2022-07-20 15:54:44 +02:00
deberta
fix train_new_from_iterator in the case of byte-level tokenizers (#17549)
2022-06-08 15:30:41 +02:00
deberta_v2
Fx support for Deberta-v[1-2], Hubert and LXMERT (#17539)
2022-06-07 18:05:20 +02:00
decision_transformer
Update expected values in DecisionTransformerModelIntegrationTest (#18016)
2022-07-05 14:53:43 +02:00
deformable_detr
[fix] Add DeformableDetrFeatureExtractor (#19140)
2022-09-22 09:45:24 +02:00
deit
Add TF DeiT implementation (#17806)
2022-07-13 18:04:08 +01:00
detr
[fix] Add DeformableDetrFeatureExtractor (#19140)
2022-09-22 09:45:24 +02:00
distilbert
…
dit
…
donut
remvoe _create_and_check_torch_fx_tracing in specific test files (#18667)
2022-09-07 16:22:09 +02:00
dpr
Replace as_target context managers by direct calls (#18325)
2022-07-29 08:09:09 -04:00
dpt
Add post_process_semantic_segmentation method to DPTFeatureExtractor (#19107)
2022-09-21 15:15:26 +03:00
electra
…
encoder_decoder
TF: use the correct config with (...)EncoderDecoder models (#18097)
2022-07-22 13:31:45 +01:00
ernie
add task_type_id to BERT to support ERNIE-2.0 and ERNIE-3.0 models (#18686)
2022-09-09 07:36:46 -04:00
flaubert
…
flava
has_attentions - consistent test skipping logic and tf tests (#17495)
2022-06-09 09:50:03 +02:00
fnet
Black preview (#17217)
2022-05-12 16:25:55 -04:00
fsmt
Not use -1e4 as attn mask (#17306)
2022-06-20 16:16:16 +02:00
funnel
Update serving code to enable saved_model=True (#18153)
2022-07-22 18:05:38 +01:00
glpn
…
gpt2
Fixing OPT fast tokenizer option. (#18753)
2022-09-15 17:12:58 +02:00
gpt_neo
fix train_new_from_iterator in the case of byte-level tokenizers (#17549)
2022-06-08 15:30:41 +02:00
gpt_neox
skip some gpt_neox tests that require 80G RAM (#17923)
2022-07-01 09:04:38 -04:00
gpt_neox_japanese
Add support for Japanese GPT-NeoX-based model by ABEJA, Inc. (#18814)
2022-09-14 10:17:40 -04:00
gptj
TF: GPT-J compatible with XLA generation (#17986)
2022-07-06 15:02:07 +01:00
groupvit
[TensorFlow] Adding GroupViT (#18020)
2022-09-29 10:48:04 +01:00
herbert
…
hubert
Fix train_step, test_step and tests for CLIP (#18684)
2022-09-09 20:01:02 +01:00
ibert
fix train_new_from_iterator in the case of byte-level tokenizers (#17549)
2022-06-08 15:30:41 +02:00
imagegpt
Enabling imageGPT auto feature extractor. (#16871)
2022-05-24 12:30:46 +02:00
layoutlm
TF: TF 2.10 unpin + related onnx test skips (#18995)
2022-09-12 19:30:27 +01:00
layoutlmv2
add warning to let the user know that the __call__ method is faster than encode + pad for a fast tokenizer (#18693)
2022-08-24 06:27:56 -04:00
layoutlmv3
Fix test_tf_encode_plus_sent_to_model for LayoutLMv3 (#18898)
2022-09-06 14:51:03 +02:00
layoutxlm
LayoutXLMProcessor: ensure 1-to-1 mapping between samples and images, and add test for it (#18774)
2022-08-30 14:43:14 +02:00
led
Fix tflongformer int dtype (#18907)
2022-09-12 17:51:10 +01:00
levit
Add skip logic for attentions test - Levit (#17633)
2022-06-10 12:46:30 +02:00
longformer
Fix tflongformer int dtype (#18907)
2022-09-12 17:51:10 +01:00
longt5
Skip test_export_to_onnx for LongT5 if torch < 1.11 (#19122)
2022-09-20 21:52:18 +02:00
luke
Adding fine-tuning models to LUKE (#18353)
2022-08-01 11:09:47 -04:00
lxmert
Update serving code to enable saved_model=True (#18153)
2022-07-22 18:05:38 +01:00
m2m_100
Replace as_target context managers by direct calls (#18325)
2022-07-29 08:09:09 -04:00
marian
Replace as_target context managers by direct calls (#18325)
2022-07-29 08:09:09 -04:00
markuplm
Add MarkupLM (#19198)
2022-09-30 08:25:43 +02:00
maskformer
Fix test_inference_instance_segmentation_head (#17872)
2022-06-24 19:36:45 +02:00
mbart
Replace as_target context managers by direct calls (#18325)
2022-07-29 08:09:09 -04:00
mbart50
Replace as_target context managers by direct calls (#18325)
2022-07-29 08:09:09 -04:00
mctct
Replace as_target context managers by direct calls (#18325)
2022-07-29 08:09:09 -04:00
megatron_bert
…
megatron_gpt2
…
mluke
Black preview (#17217)
2022-05-12 16:25:55 -04:00
mobilebert
Fix train_step, test_step and tests for CLIP (#18684)
2022-09-09 20:01:02 +01:00
mobilevit
Add semantic segmentation post-processing method to MobileViT (#19105)
2022-09-23 16:24:28 +03:00
mpnet
…
mt5
Fix expected loss values in some (m)T5 tests (#18177)
2022-07-18 15:26:21 +02:00
mvp
Replace as_target context managers by direct calls (#18325)
2022-07-29 08:09:09 -04:00
nezha
speed up test (#18106)
2022-07-12 04:28:28 -04:00
nllb
Replace as_target context managers by direct calls (#18325)
2022-07-29 08:09:09 -04:00
nystromformer
…
openai
…
opt
add missing require_tf for TFOPTGenerationTest (#19010)
2022-09-13 18:10:11 +02:00
owlvit
fix owlvit tests, update docstring examples (#18586)
2022-08-11 19:10:25 +03:00
pegasus
Replace as_target context managers by direct calls (#18325)
2022-07-29 08:09:09 -04:00
pegasus_x
Fix CI for PegasusX (#19025)
2022-09-14 14:45:00 +02:00
perceiver
Replace as_target context managers by direct calls (#18325)
2022-07-29 08:09:09 -04:00
phobert
…
plbart
Replace as_target context managers by direct calls (#18325)
2022-07-29 08:09:09 -04:00
poolformer
has_attentions - consistent test skipping logic and tf tests (#17495)
2022-06-09 09:50:03 +02:00
prophetnet
Black preview (#17217)
2022-05-12 16:25:55 -04:00
qdqbert
…
rag
Avoid GPU OOM for a TF Rag test (#17638)
2022-06-10 18:50:29 +02:00
realm
Black preview (#17217)
2022-05-12 16:25:55 -04:00
reformer
Black preview (#17217)
2022-05-12 16:25:55 -04:00
regnet
Run tests if skip condition not met (#18764)
2022-08-30 14:03:28 +02:00
rembert
…
resnet
FX support for ConvNext, Wav2Vec2 and ResNet (#19053)
2022-09-16 10:57:41 +02:00
retribert
fix retribert's test_torch_encode_plus_sent_to_model (#17231)
2022-05-17 14:33:13 +02:00
roberta
fix train_new_from_iterator in the case of byte-level tokenizers (#17549)
2022-06-08 15:30:41 +02:00
roformer
…
segformer
Add post_process_semantic_segmentation method to SegFormer (#19072)
2022-09-21 11:40:35 +03:00
sew
…
sew_d
…
speech_encoder_decoder
send model to the correct device (#18800)
2022-08-29 18:46:30 +02:00
speech_to_text
remvoe _create_and_check_torch_fx_tracing in specific test files (#18667)
2022-09-07 16:22:09 +02:00
speech_to_text_2
Fx support for multiple model architectures (#17393)
2022-05-31 10:02:55 +02:00
splinter
Fix Splinter test (#17854)
2022-06-24 16:26:14 +02:00
squeezebert
…
swin
remvoe _create_and_check_torch_fx_tracing in specific test files (#18667)
2022-09-07 16:22:09 +02:00
swinv2
Add swin transformer v2 (#17469)
2022-07-27 11:14:47 -04:00
t5
Replace as_target context managers by direct calls (#18325)
2022-07-29 08:09:09 -04:00
tapas
Fix train_step, test_step and tests for CLIP (#18684)
2022-09-09 20:01:02 +01:00
tapex
Replace as_target context managers by direct calls (#18325)
2022-07-29 08:09:09 -04:00
trajectory_transformer
Add trajectory transformer (#17141)
2022-05-17 19:07:43 -04:00
transfo_xl
Add magic method to our TF models to convert datasets with column inference (#17160)
2022-06-06 15:53:49 +01:00
trocr
Fx support for multiple model architectures (#17393)
2022-05-31 10:02:55 +02:00
unispeech
…
unispeech_sat
…
van
has_attentions - consistent test skipping logic and tf tests (#17495)
2022-06-09 09:50:03 +02:00
videomae
Use repo_type instead of deprecated datasets repo IDs (#19202)
2022-09-26 09:50:48 -04:00
vilt
Add accelerate support for ViLT (#18683)
2022-09-22 13:14:39 +02:00
vision_encoder_decoder
send model to the correct device (#18800)
2022-08-29 18:46:30 +02:00
vision_text_dual_encoder
…
visual_bert
…
vit
Improve vision models (#17731)
2022-06-24 11:34:51 +02:00
vit_mae
TF: tests for (de)serializable models with resized tokens (#19013)
2022-09-16 16:38:08 +01:00
vit_msn
fix: ckpt paths. (#19159)
2022-09-22 11:03:01 -04:00
wav2vec2
FX support for ConvNext, Wav2Vec2 and ResNet (#19053)
2022-09-16 10:57:41 +02:00
wav2vec2_conformer
[Test] Fix W2V-Conformer integration test (#17303)
2022-05-17 18:20:36 +02:00
wav2vec2_phoneme
…
wav2vec2_with_lm
[Wav2vec2 + LM Test] Improve wav2vec2 with lm tests and make torch version dependent for now (#18749)
2022-08-26 14:11:55 +02:00
wavlm
…
x_clip
Use repo_type instead of deprecated datasets repo IDs (#19202)
2022-09-26 09:50:48 -04:00
xglm
XGLM - Fix Softmax NaNs when using FP16 (#18057)
2022-09-29 10:42:07 +02:00
xlm
…
xlm_prophetnet
Black preview (#17217)
2022-05-12 16:25:55 -04:00
xlm_roberta
Black preview (#17217)
2022-05-12 16:25:55 -04:00
xlm_roberta_xl
…
xlnet
Return scalar losses instead of per-sample means (#18013)
2022-07-04 17:26:19 +01:00
yolos
[fix] Add DeformableDetrFeatureExtractor (#19140)
2022-09-22 09:45:24 +02:00
yoso
fix train_new_from_iterator in the case of byte-level tokenizers (#17549)
2022-06-08 15:30:41 +02:00
__init__.py
…