Tanmay Laud f7f872955d Big Bird Fast Tokenizer implementation (#11075) ...

* Added Big Bird Fast Tokenizer initial file

* style fixes

* flake fixes

* Added big bird fast tokenizer to init files

* Added big bird fast to Auto tokenization

* fix styles

* minor quality fixes

* Added initial test code

* Fix SpmConverter when precompiled_charsmap doesn't exist

* fixed post processor

* minor style fix

* minor fix input names

* Actually fix identity normalization

* style

* Added token type ids to fast tokenizer

* style

* flake fix

* fix copies

Co-authored-by: Anthony MOI <m.anthony.moi@gmail.com>

2021-05-10 03:01:23 -04:00

..

albert.rst

Honor contributors to models (#11329)

2021-04-21 09:47:27 -04:00

auto.rst

Add the ImageClassificationPipeline (#11598)

2021-05-07 08:08:40 -04:00

bart.rst

Examples reorg (#11350)

2021-04-21 11:11:20 -04:00

barthez.rst

Examples reorg (#11350)

2021-04-21 11:11:20 -04:00

bert_japanese.rst

Honor contributors to models (#11329)

2021-04-21 09:47:27 -04:00

bert.rst

Honor contributors to models (#11329)

2021-04-21 09:47:27 -04:00

bertgeneration.rst

Honor contributors to models (#11329)

2021-04-21 09:47:27 -04:00

bertweet.rst

Honor contributors to models (#11329)

2021-04-21 09:47:27 -04:00

bigbird_pegasus.rst

Add BigBirdPegasus (#10991)

2021-05-07 09:27:43 +02:00

bigbird.rst

Big Bird Fast Tokenizer implementation (#11075)

2021-05-10 03:01:23 -04:00

blenderbot_small.rst

Honor contributors to models (#11329)

2021-04-21 09:47:27 -04:00

blenderbot.rst

Honor contributors to models (#11329)

2021-04-21 09:47:27 -04:00

bort.rst

Honor contributors to models (#11329)

2021-04-21 09:47:27 -04:00

camembert.rst

Honor contributors to models (#11329)

2021-04-21 09:47:27 -04:00

convbert.rst

Honor contributors to models (#11329)

2021-04-21 09:47:27 -04:00

cpm.rst

Honor contributors to models (#11329)

2021-04-21 09:47:27 -04:00

ctrl.rst

Honor contributors to models (#11329)

2021-04-21 09:47:27 -04:00

deberta_v2.rst

Honor contributors to models (#11329)

2021-04-21 09:47:27 -04:00

deberta.rst

Implement Fast Tokenization for Deberta (#11387)

2021-04-30 08:08:15 -04:00

deit.rst

Honor contributors to models (#11329)

2021-04-21 09:47:27 -04:00

dialogpt.rst

ADD BORT (#9813)

2021-01-27 21:25:11 +03:00

distilbert.rst

Examples reorg (#11350)

2021-04-21 11:11:20 -04:00

dpr.rst

Honor contributors to models (#11329)

2021-04-21 09:47:27 -04:00

electra.rst

[Flax] Add Electra models (#11426)

2021-05-04 20:56:09 +02:00

encoderdecoder.rst

Copyright (#8970)

2020-12-07 18:36:34 -05:00

flaubert.rst

Honor contributors to models (#11329)

2021-04-21 09:47:27 -04:00

fsmt.rst

Honor contributors to models (#11329)

2021-04-21 09:47:27 -04:00

funnel.rst

Honor contributors to models (#11329)

2021-04-21 09:47:27 -04:00

gpt2.rst

Honor contributors to models (#11329)

2021-04-21 09:47:27 -04:00

gpt_neo.rst

Honor contributors to models (#11329)

2021-04-21 09:47:27 -04:00

gpt.rst

Honor contributors to models (#11329)

2021-04-21 09:47:27 -04:00

herbert.rst

Honor contributors to models (#11329)

2021-04-21 09:47:27 -04:00

ibert.rst

Honor contributors to models (#11329)

2021-04-21 09:47:27 -04:00

layoutlm.rst

Honor contributors to models (#11329)

2021-04-21 09:47:27 -04:00

led.rst

Honor contributors to models (#11329)

2021-04-21 09:47:27 -04:00

longformer.rst

Honor contributors to models (#11329)

2021-04-21 09:47:27 -04:00

luke.rst

Add LUKE (#11223)

2021-05-03 09:07:29 -04:00

lxmert.rst

Honor contributors to models (#11329)

2021-04-21 09:47:27 -04:00

m2m_100.rst

Honor contributors to models (#11329)

2021-04-21 09:47:27 -04:00

marian.rst

Honor contributors to models (#11329)

2021-04-21 09:47:27 -04:00

mbart.rst

Honor contributors to models (#11329)

2021-04-21 09:47:27 -04:00

megatron_bert.rst

Honor contributors to models (#11329)

2021-04-21 09:47:27 -04:00

megatron_gpt2.rst

Honor contributors to models (#11329)

2021-04-21 09:47:27 -04:00

mobilebert.rst

Honor contributors to models (#11329)

2021-04-21 09:47:27 -04:00

mpnet.rst

MPNet copyright files (#9015)

2020-12-10 09:29:38 -05:00

mt5.rst

Honor contributors to models (#11329)

2021-04-21 09:47:27 -04:00

pegasus.rst

Examples reorg (#11350)

2021-04-21 11:11:20 -04:00

phobert.rst

Honor contributors to models (#11329)

2021-04-21 09:47:27 -04:00

prophetnet.rst

Copyright (#8970)

2020-12-07 18:36:34 -05:00

rag.rst

Honor contributors to models (#11329)

2021-04-21 09:47:27 -04:00

reformer.rst

Honor contributors to models (#11329)

2021-04-21 09:47:27 -04:00

retribert.rst

Examples reorg (#11350)

2021-04-21 11:11:20 -04:00

roberta.rst

[FlaxRoberta] Add FlaxRobertaModels & adapt run_mlm_flax.py (#11470)

2021-05-04 19:57:59 +02:00

speech_to_text.rst

Honor contributors to models (#11329)

2021-04-21 09:47:27 -04:00

squeezebert.rst

Honor contributors to models (#11329)

2021-04-21 09:47:27 -04:00

t5.rst

Honor contributors to models (#11329)

2021-04-21 09:47:27 -04:00

tapas.rst

Honor contributors to models (#11329)

2021-04-21 09:47:27 -04:00

transformerxl.rst

Honor contributors to models (#11329)

2021-04-21 09:47:27 -04:00

vit.rst

Honor contributors to models (#11329)

2021-04-21 09:47:27 -04:00

wav2vec2.rst

Honor contributors to models (#11329)

2021-04-21 09:47:27 -04:00

xlm.rst

Honor contributors to models (#11329)

2021-04-21 09:47:27 -04:00

xlmprophetnet.rst

Copyright (#8970)

2020-12-07 18:36:34 -05:00

xlmroberta.rst

Honor contributors to models (#11329)

2021-04-21 09:47:27 -04:00

xlnet.rst

Examples reorg (#11350)

2021-04-21 11:11:20 -04:00

xlsr_wav2vec2.rst

[XLSR-Wav2Vec2] Add multi-lingual Wav2Vec2 models (#10648)

2021-03-11 17:44:18 +03:00