SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity
Files
3a8de58c5192b620228128430ea52e6eda81c40a
HuggingFace_transformer/docs/source/model_doc
History
Sidd Karamcheti 3a8de58c51 Add Mistral GPT-2 Stability Tweaks (#13573) 
* Add layer-wise scaling

* Add reorder & upcasting argument

* Add OpenAI GPT-2 weight initialization scheme

* start `layer_idx` count at zero for consistency

* disentangle attn and reordered and upscaled attn function

* rename `scale_attn_by_layer` to `scale_attn_by_layer_id`

* make autocast from amp compatible with pytorch<1.6

* fix docstring

* style fixes

* Add fixes from PR feedback, style tweaks

* Fix doc whitespace

* Reformat

* First pass scale_attn_by_layer_idx and reorder_and_upcast_attn tests

* Rename scale_attn_by_layer_idx, add tip

* Remove extra newline

* add test for weight initialization

* update code format

* add assert check weights are fp32

* remove assert

* Fix incorrect merge

* Fix shape mismatch in baddbmm

* Add generation test for Mistral flags

Co-authored-by: leandro <leandro.vonwerra@spoud.io>
Co-authored-by: Keshav Santhanam <keshav2@stanford.edu>
Co-authored-by: J38 <jebolton@stanford.edu>
2021-10-04 07:37:09 -04:00
..
albert.rst
albert flax (#13294)
2021-08-30 17:29:27 +02:00
auto.rst
Add Speech AutoModels (#13655)
2021-09-21 08:50:33 +02:00
bart.rst
FlaxBart (#11537)
2021-06-14 15:16:08 +05:30
barthez.rst
Examples reorg (#11350)
2021-04-21 11:11:20 -04:00
beit.rst
beit-flax (#13515)
2021-09-21 13:34:19 +02:00
bert_japanese.rst
Honor contributors to models (#11329)
2021-04-21 09:47:27 -04:00
bert.rst
[Flax] Correct flax docs (#12782)
2021-08-04 16:31:23 +02:00
bertgeneration.rst
Honor contributors to models (#11329)
2021-04-21 09:47:27 -04:00
bertweet.rst
Honor contributors to models (#11329)
2021-04-21 09:47:27 -04:00
bigbird_pegasus.rst
Add BigBirdPegasus (#10991)
2021-05-07 09:27:43 +02:00
bigbird.rst
Flax Big Bird (#11967)
2021-06-14 20:01:03 +01:00
blenderbot_small.rst
Add BlenderBot small tokenizer to the init (#13367)
2021-09-22 19:00:47 -04:00
blenderbot.rst
Honor contributors to models (#11329)
2021-04-21 09:47:27 -04:00
bort.rst
Honor contributors to models (#11329)
2021-04-21 09:47:27 -04:00
byt5.rst
Improve T5 docs (#13240)
2021-09-01 15:05:40 +02:00
camembert.rst
Honor contributors to models (#11329)
2021-04-21 09:47:27 -04:00
canine.rst
Wrong model is used in example, should be character instead of subword model (#12676)
2021-07-13 08:40:27 -04:00
clip.rst
add and fix examples (#12810)
2021-07-20 09:28:50 -04:00
convbert.rst
Honor contributors to models (#11329)
2021-04-21 09:47:27 -04:00
cpm.rst
Honor contributors to models (#11329)
2021-04-21 09:47:27 -04:00
ctrl.rst
Honor contributors to models (#11329)
2021-04-21 09:47:27 -04:00
deberta_v2.rst
Deberta_v2 tf (#13120)
2021-08-31 06:32:47 -04:00
deberta.rst
Deberta tf (#12972)
2021-08-12 05:01:26 -04:00
deit.rst
Honor contributors to models (#11329)
2021-04-21 09:47:27 -04:00
detr.rst
Improve detr (#12147)
2021-06-17 10:37:54 -04:00
dialogpt.rst
ADD BORT (#9813)
2021-01-27 21:25:11 +03:00
distilbert.rst
Fix typo distilbert doc (#13643)
2021-09-20 15:10:33 -04:00
dpr.rst
[DPR] Correct init (#13796)
2021-09-30 18:55:20 +02:00
electra.rst
[Flax] Add Electra models (#11426)
2021-05-04 20:56:09 +02:00
encoderdecoder.rst
Make Flax GPT2 working with cross attention (#13008)
2021-08-23 17:57:29 +02:00
flaubert.rst
Honor contributors to models (#11329)
2021-04-21 09:47:27 -04:00
fnet.rst
Add FNet (#13045)
2021-09-20 13:24:30 +02:00
fsmt.rst
Honor contributors to models (#11329)
2021-04-21 09:47:27 -04:00
funnel.rst
Honor contributors to models (#11329)
2021-04-21 09:47:27 -04:00
gpt2.rst
Add Mistral GPT-2 Stability Tweaks (#13573)
2021-10-04 07:37:09 -04:00
gpt_neo.rst
FlaxGPTNeo (#12493)
2021-07-06 18:55:18 +05:30
gpt.rst
Honor contributors to models (#11329)
2021-04-21 09:47:27 -04:00
gptj.rst
[docs/gpt-j] fix typo (#13851)
2021-10-04 12:30:50 +02:00
herbert.rst
Honor contributors to models (#11329)
2021-04-21 09:47:27 -04:00
hubert.rst
Add Wav2Vec2 & Hubert ForSequenceClassification (#13153)
2021-08-27 20:52:51 +03:00
ibert.rst
Honor contributors to models (#11329)
2021-04-21 09:47:27 -04:00
layoutlm.rst
Honor contributors to models (#11329)
2021-04-21 09:47:27 -04:00
layoutlmv2.rst
Add LayoutLMv2 + LayoutXLM (#12604)
2021-08-30 12:35:42 +02:00
layoutxlm.rst
Add tokenizer docs (#13373)
2021-09-02 09:46:05 +02:00
led.rst
Make gradient_checkpointing a training argument (#13657)
2021-09-22 07:51:38 -04:00
longformer.rst
Honor contributors to models (#11329)
2021-04-21 09:47:27 -04:00
luke.rst
Add LUKE (#11223)
2021-05-03 09:07:29 -04:00
lxmert.rst
Honor contributors to models (#11329)
2021-04-21 09:47:27 -04:00
m2m_100.rst
replace tgt_lang by tgt_text (#13061)
2021-08-09 22:47:05 +05:30
marian.rst
Rely on huggingface_hub for common tools (#13100)
2021-08-12 14:59:02 +02:00
mbart.rst
fix example (#13387)
2021-09-02 11:32:18 +02:00
megatron_bert.rst
Honor contributors to models (#11329)
2021-04-21 09:47:27 -04:00
megatron_gpt2.rst
Honor contributors to models (#11329)
2021-04-21 09:47:27 -04:00
mobilebert.rst
Honor contributors to models (#11329)
2021-04-21 09:47:27 -04:00
mpnet.rst
MPNet copyright files (#9015)
2020-12-10 09:29:38 -05:00
mt5.rst
Fix mT5 documentation (#13639)
2021-09-20 07:53:31 -04:00
pegasus.rst
[Flax] Addition of FlaxPegasus (#13420)
2021-09-14 17:15:19 +02:00
phobert.rst
Honor contributors to models (#11329)
2021-04-21 09:47:27 -04:00
prophetnet.rst
Copyright (#8970)
2020-12-07 18:36:34 -05:00
rag.rst
Honor contributors to models (#11329)
2021-04-21 09:47:27 -04:00
reformer.rst
Honor contributors to models (#11329)
2021-04-21 09:47:27 -04:00
rembert.rst
Add RemBERT model code to huggingface (#10692)
2021-07-24 11:31:42 -04:00
retribert.rst
Examples reorg (#11350)
2021-04-21 11:11:20 -04:00
roberta.rst
[FlaxRoberta] Add FlaxRobertaModels & adapt run_mlm_flax.py (#11470)
2021-05-04 19:57:59 +02:00
roformer.rst
[RoFormer] Fix some issues (#12397)
2021-07-06 03:31:57 -04:00
speech_to_text_2.rst
Add SpeechEncoderDecoder & Speech2Text2 (#13186)
2021-09-01 13:33:31 +02:00
speech_to_text.rst
fix: typo spelling grammar (#13212)
2021-08-30 08:09:14 -04:00
speechencoderdecoder.rst
Add SpeechEncoderDecoder & Speech2Text2 (#13186)
2021-09-01 13:33:31 +02:00
splinter.rst
Add splinter (#12955)
2021-08-17 08:29:01 -04:00
squeezebert.rst
Honor contributors to models (#11329)
2021-04-21 09:47:27 -04:00
t5.rst
Improve T5 docs (#13240)
2021-09-01 15:05:40 +02:00
t5v1.1.rst
Improve T5 docs (#13240)
2021-09-01 15:05:40 +02:00
tapas.rst
Honor contributors to models (#11329)
2021-04-21 09:47:27 -04:00
transformerxl.rst
Honor contributors to models (#11329)
2021-04-21 09:47:27 -04:00
visual_bert.rst
Fix VisualBERT docs (#13106)
2021-08-13 11:44:04 +05:30
vit.rst
Add DINO conversion script (#13265)
2021-08-26 17:25:20 +02:00
wav2vec2.rst
Add Wav2Vec2 & Hubert ForSequenceClassification (#13153)
2021-08-27 20:52:51 +03:00
xlm.rst
Honor contributors to models (#11329)
2021-04-21 09:47:27 -04:00
xlmprophetnet.rst
Copyright (#8970)
2020-12-07 18:36:34 -05:00
xlmroberta.rst
Honor contributors to models (#11329)
2021-04-21 09:47:27 -04:00
xlnet.rst
Examples reorg (#11350)
2021-04-21 11:11:20 -04:00
xlsr_wav2vec2.rst
[XLSR-Wav2Vec2] Add multi-lingual Wav2Vec2 models (#10648)
2021-03-11 17:44:18 +03:00