SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity
Files
76d13de5ae5bab9fefcbe3db72fb4e44b57c0c3c
HuggingFace_transformer/docs/source/en/model_doc
History
Matt ee0d001de7 Add a TF in-graph tokenizer for BERT (#17701) 
* Add a TF in-graph tokenizer for BERT

* Add from_pretrained

* Add proper truncation, option handling to match other tokenizers

* Add proper imports and guards

* Add test, fix all the bugs exposed by said test

* Fix truncation of paired texts in graph mode, more test updates

* Small fixes, add a (very careful) test for savedmodel

* Add tensorflow-text dependency, make fixup

* Update documentation

* Update documentation

* make fixup

* Slight changes to tests

* Add some docstring examples

* Update tests

* Update tests and add proper lowercasing/normalization

* make fixup

* Add docstring for padding!

* Mark slow tests

* make fixup

* Fall back to BertTokenizerFast if BertTokenizer is unavailable

* Fall back to BertTokenizerFast if BertTokenizer is unavailable

* make fixup

* Properly handle tensorflow-text dummies
2022-06-27 12:06:21 +01:00
..
albert.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
auto.mdx
Add Visual Question Answering (VQA) pipeline (#17286)
2022-06-13 07:49:44 -04:00
bart.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
barthez.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
bartpho.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
beit.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
bert-generation.mdx
Result of new doc style with fixes (#17015)
2022-04-29 17:42:15 -04:00
bert-japanese.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
bert.mdx
Add a TF in-graph tokenizer for BERT (#17701)
2022-06-27 12:06:21 +01:00
bertweet.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
big_bird.mdx
[FlaxBert] Add ForCausalLM (#16995)
2022-05-03 11:26:19 +02:00
bigbird_pegasus.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
blenderbot-small.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
blenderbot.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
bloom.mdx
Add BloomForSequenceClassification and BloomForTokenClassification classes (#17639)
2022-06-14 17:10:12 +02:00
bort.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
byt5.mdx
[Doctests] Fix all T5 doc tests (#16646)
2022-04-13 11:36:54 +02:00
camembert.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
canine.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
clip.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
codegen.mdx
Add CodeGen model (#17443)
2022-06-24 17:10:38 +02:00
convbert.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
convnext.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
cpm.mdx
Allow all imports from transformers (#17050)
2022-05-02 12:47:39 -04:00
ctrl.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
cvt.mdx
Add CvT (#17299)
2022-05-18 17:47:18 +02:00
data2vec.mdx
Add TFData2VecVision for semantic segmentation (#17271)
2022-06-08 14:03:18 +01:00
deberta-v2.mdx
Add DebertaV2ForMultipleChoice (#17135)
2022-05-10 16:21:44 -04:00
deberta.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
decision_transformer.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
deit.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
detr.mdx
Fix Automatic Download of Pretrained Weights in DETR (#17712)
2022-06-21 16:45:35 +02:00
dialogpt.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
distilbert.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
dit.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
dpr.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
dpt.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
electra.mdx
[FlaxBert] Add ForCausalLM (#16995)
2022-05-03 11:26:19 +02:00
encoder-decoder.mdx
Improve encoder decoder model docs (#17815)
2022-06-24 14:48:19 +02:00
flaubert.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
flava.mdx
[feat] Add FLAVA model (#16654)
2022-05-11 14:56:48 -07:00
fnet.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
fsmt.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
funnel.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
glpn.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
gpt2.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
gpt_neo.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
gpt_neox.mdx
[WIP] Adding GPT-NeoX-20B (#16659)
2022-05-24 09:31:10 -04:00
gptj.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
herbert.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
hubert.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
ibert.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
imagegpt.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
layoutlm.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
layoutlmv2.mdx
Correct & Improve Doctests for LayoutLMv2 (#17168)
2022-05-23 08:02:31 -04:00
layoutlmv3.mdx
Add LayoutLMv3 (#17060)
2022-05-24 09:53:45 +02:00
layoutxlm.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
led.mdx
[LED] fix global_attention_mask not being passed for generation and docs clarification about grad checkpointing (#17112)
2022-05-17 23:44:37 +02:00
levit.mdx
Adding LeViT Model by Facebook (#17466)
2022-06-01 17:06:20 +02:00
longformer.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
longt5.mdx
Add LongT5 model (#16792)
2022-06-13 22:36:58 +02:00
luke.mdx
Result of new doc style with fixes (#17015)
2022-04-29 17:42:15 -04:00
lxmert.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
m2m_100.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
marian.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
maskformer.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
mbart.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
mctct.mdx
M-CTC-T Model (#16402)
2022-06-08 00:33:07 +02:00
megatron_gpt2.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
megatron-bert.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
mluke.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
mobilebert.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
mpnet.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
mt5.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
nezha.mdx
Nezha Pytorch implementation (#17776)
2022-06-23 12:36:22 -04:00
nystromformer.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
openai-gpt.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
opt.mdx
Opt in flax and tf (#17388)
2022-05-31 18:41:22 +02:00
pegasus.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
perceiver.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
phobert.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
plbart.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
poolformer.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
prophetnet.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
qdqbert.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
rag.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
realm.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
reformer.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
regnet.mdx
RegNet (#16188)
2022-04-07 21:58:00 +02:00
rembert.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
resnet.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
retribert.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
roberta.mdx
[FlaxBert] Add ForCausalLM (#16995)
2022-05-03 11:26:19 +02:00
roformer.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
segformer.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
sew-d.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
sew.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
speech_to_text_2.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
speech_to_text.mdx
[Speech2Text Doc] Fix docs (#16611)
2022-04-06 14:19:00 +02:00
speech-encoder-decoder.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
splinter.mdx
Add support for pretraining recurring span selection to Splinter (#17247)
2022-05-17 23:42:14 +02:00
squeezebert.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
swin.mdx
Add Tensorflow Swin model (#16988)
2022-05-16 22:19:53 +01:00
t5.mdx
[DocTests] Fix some doc tests (#16889)
2022-04-23 08:40:14 +02:00
t5v1.1.mdx
[Doctests] Fix all T5 doc tests (#16646)
2022-04-13 11:36:54 +02:00
tapas.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
tapex.mdx
Add TAPEX (#16473)
2022-04-08 10:57:51 +02:00
trajectory_transformer.mdx
Add trajectory transformer (#17141)
2022-05-17 19:07:43 -04:00
transfo-xl.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
trocr.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
ul2.mdx
Add UL2 (just docs) (#17740)
2022-06-21 10:24:50 +02:00
unispeech-sat.mdx
Add Wav2Vec2Conformer (#16812)
2022-05-17 00:43:16 +02:00
unispeech.mdx
Add Wav2Vec2Conformer (#16812)
2022-05-17 00:43:16 +02:00
van.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
vilt.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
vision-encoder-decoder.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
vision-text-dual-encoder.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
visual_bert.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
vit_mae.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
vit.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
wav2vec2_phoneme.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
wav2vec2-conformer.mdx
[Wav2Vec2Conformer] Official release (#17709)
2022-06-15 18:34:15 +02:00
wav2vec2.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
wavlm.mdx
Add Wav2Vec2Conformer (#16812)
2022-05-17 00:43:16 +02:00
xglm.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
xlm-prophetnet.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
xlm-roberta-xl.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
xlm-roberta.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
xlm.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
xlnet.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
xls_r.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
xlsr_wav2vec2.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00
yolos.mdx
Add YOLOS (#16848)
2022-05-02 18:30:55 +02:00
yoso.mdx
Enable doc in Spanish (#16518)
2022-04-04 10:25:46 -04:00