Rémi Louf f8e98d6779 load pretrained embeddings in Bert decoder ...

In Rothe et al.'s "Leveraging Pre-trained Checkpoints for Sequence
Generation Tasks", Bert2Bert is initialized with pre-trained weights for
the encoder, and only pre-trained embeddings for the decoder. The
current version of the code completely randomizes the weights of the
decoder.

We write a custom function to initiliaze the weights of the decoder; we
first initialize the decoder with the weights and then randomize
everything but the embeddings.

2019-10-11 16:48:11 +02:00

..

data

Add docstring for processor method

2019-09-27 17:32:28 +02:00

tests

add test for initialization of Bert2Rnd

2019-10-10 18:07:11 +02:00

__init__.py

rename Bert2Bert -> Bert2Rnd

2019-10-08 16:30:58 +02:00

__main__.py

[BIG] pytorch-transformers => transformers

2019-09-26 10:15:53 +02:00

configuration_auto.py

[BIG] pytorch-transformers => transformers

2019-09-26 10:15:53 +02:00

configuration_bert.py

[BIG] pytorch-transformers => transformers

2019-09-26 10:15:53 +02:00

configuration_distilbert.py

[BIG] pytorch-transformers => transformers

2019-09-26 10:15:53 +02:00

configuration_gpt2.py

update links to new weights

2019-10-03 10:27:11 -04:00

configuration_openai.py

Merge branch 'master' into tf2

2019-09-26 12:02:54 +02:00

configuration_roberta.py

[BIG] pytorch-transformers => transformers

2019-09-26 10:15:53 +02:00

configuration_transfo_xl.py

[BIG] pytorch-transformers => transformers

2019-09-26 10:15:53 +02:00

configuration_utils.py

[BIG] pytorch-transformers => transformers

2019-09-26 10:15:53 +02:00

configuration_xlm.py

[BIG] pytorch-transformers => transformers

2019-09-26 10:15:53 +02:00

configuration_xlnet.py

[BIG] pytorch-transformers => transformers

2019-09-26 10:15:53 +02:00

convert_bert_original_tf_checkpoint_to_pytorch.py

[BIG] pytorch-transformers => transformers

2019-09-26 10:15:53 +02:00

convert_bert_pytorch_checkpoint_to_original_tf.py

[BIG] pytorch-transformers => transformers

2019-09-26 10:15:53 +02:00

convert_gpt2_original_tf_checkpoint_to_pytorch.py

[BIG] pytorch-transformers => transformers

2019-09-26 10:15:53 +02:00

convert_openai_original_tf_checkpoint_to_pytorch.py

[BIG] pytorch-transformers => transformers

2019-09-26 10:15:53 +02:00

convert_pytorch_checkpoint_to_tf2.py

fix buf in convert_pt_chkpt_to_tf2

2019-10-03 10:27:11 -04:00

convert_roberta_original_pytorch_checkpoint_to_pytorch.py

[BIG] pytorch-transformers => transformers

2019-09-26 10:15:53 +02:00

convert_transfo_xl_original_tf_checkpoint_to_pytorch.py

[BIG] pytorch-transformers => transformers

2019-09-26 10:15:53 +02:00

convert_xlm_original_pytorch_checkpoint_to_pytorch.py

[BIG] pytorch-transformers => transformers

2019-09-26 10:15:53 +02:00

convert_xlnet_original_tf_checkpoint_to_pytorch.py

[BIG] pytorch-transformers => transformers

2019-09-26 10:15:53 +02:00

file_utils.py

[BIG] pytorch-transformers => transformers

2019-09-26 10:15:53 +02:00

modeling_auto.py

[BIG] pytorch-transformers => transformers

2019-09-26 10:15:53 +02:00

modeling_bert.py

load pretrained embeddings in Bert decoder

2019-10-11 16:48:11 +02:00

modeling_distilbert.py

DistillBert Documentation Code Example fixes

2019-10-03 15:51:33 -04:00

modeling_gpt2.py

update links to new weights

2019-10-03 10:27:11 -04:00

modeling_openai.py

[BIG] pytorch-transformers => transformers

2019-09-26 10:15:53 +02:00

modeling_roberta.py

fix padding_idx of RoBERTa model

2019-09-27 19:03:55 -04:00

modeling_tf_auto.py

[BIG] pytorch-transformers => transformers

2019-09-26 10:15:53 +02:00

modeling_tf_bert.py

initialy -> initially

2019-10-02 15:04:18 +00:00

modeling_tf_distilbert.py

DistillBert Documentation Code Example fixes

2019-10-03 15:51:33 -04:00

modeling_tf_gpt2.py

update links to new weights

2019-10-03 10:27:11 -04:00

modeling_tf_openai.py

[BIG] pytorch-transformers => transformers

2019-09-26 10:15:53 +02:00

modeling_tf_pytorch_utils.py

[BIG] pytorch-transformers => transformers

2019-09-26 10:15:53 +02:00

modeling_tf_roberta.py

[BIG] pytorch-transformers => transformers

2019-09-26 10:15:53 +02:00

modeling_tf_transfo_xl_utilities.py

[BIG] pytorch-transformers => transformers

2019-09-26 10:15:53 +02:00

modeling_tf_transfo_xl.py

[BIG] pytorch-transformers => transformers

2019-09-26 10:15:53 +02:00

modeling_tf_utils.py

[BIG] pytorch-transformers => transformers

2019-09-26 10:15:53 +02:00

modeling_tf_xlm.py

initialy -> initially

2019-10-02 15:04:18 +00:00

modeling_tf_xlnet.py

[BIG] pytorch-transformers => transformers

2019-09-26 10:15:53 +02:00

modeling_transfo_xl_utilities.py

[BIG] pytorch-transformers => transformers

2019-09-26 10:15:53 +02:00

modeling_transfo_xl.py

[BIG] pytorch-transformers => transformers

2019-09-26 10:15:53 +02:00

modeling_utils.py

add comment on recursive weights loading

2019-10-10 10:02:03 +02:00

modeling_xlm.py

[BIG] pytorch-transformers => transformers

2019-09-26 10:15:53 +02:00

modeling_xlnet.py

[BIG] pytorch-transformers => transformers

2019-09-26 10:15:53 +02:00

optimization.py

[BIG] pytorch-transformers => transformers

2019-09-26 10:15:53 +02:00

tokenization_auto.py

[BIG] pytorch-transformers => transformers

2019-09-26 10:15:53 +02:00

tokenization_bert.py

[BIG] pytorch-transformers => transformers

2019-09-26 10:15:53 +02:00

tokenization_distilbert.py

[BIG] pytorch-transformers => transformers

2019-09-26 10:15:53 +02:00

tokenization_gpt2.py

update links to new weights

2019-10-03 10:27:11 -04:00

tokenization_openai.py

[BIG] pytorch-transformers => transformers

2019-09-26 10:15:53 +02:00

tokenization_roberta.py

Close #1304

2019-09-27 17:02:53 -04:00

tokenization_transfo_xl.py

[BIG] pytorch-transformers => transformers

2019-09-26 10:15:53 +02:00

tokenization_utils.py

Decode documentaton

2019-10-04 14:27:38 -04:00

tokenization_xlm.py

[BIG] pytorch-transformers => transformers

2019-09-26 10:15:53 +02:00

tokenization_xlnet.py

Merge branch 'master' into tf2

2019-09-26 12:02:54 +02:00