Commit Graph

26 Commits

Author SHA1 Message Date
Julien Chaumond
bdfe21ab24 Change param order for consistency 2019-11-26 13:08:12 -05:00
LysandreJik
c536c2a480 ALBERT Input Embeds 2019-11-26 13:08:12 -05:00
Lysandre
c9cb7f8a0f Torch 1.1.0 compatibility + FP16 O1 + TF checkpoints
Co-authored-by: wassname
2019-11-26 13:08:12 -05:00
Lysandre
d9daad98c7 Re-ordering of group_idx/layer_idx + Python 2 tests 2019-11-26 13:08:12 -05:00
Lysandre
16263f9685 Headmasking 2019-11-26 13:08:12 -05:00
Lysandre
abb23a78ba Head pruning for ALBERT 2019-11-26 13:08:12 -05:00
Lysandre
4374eaea78 ALBERT for SQuAD 2019-11-26 13:08:12 -05:00
Lysandre
70d99980de ALBERT-V2 2019-11-26 13:08:12 -05:00
Lysandre
6637a77f80 AlbertForSequenceClassification 2019-11-26 13:08:12 -05:00
Lysandre
4f3a54bfc8 ALBERT can load pre-trained models. Doesn't inherit from BERT anymore. 2019-11-26 13:08:12 -05:00
Lysandre
c4403006b8 External MLM head 2019-11-26 13:08:12 -05:00
Lysandre
b21402fc86 Python 2 tests + licence 2019-11-26 13:08:12 -05:00
Lysandre
c14a22272f ALBERT passes all tests 2019-11-26 13:08:12 -05:00
Lysandre
870320a24e Early tests 2019-11-26 13:08:12 -05:00
Lysandre
25a31953e8 Output Attentions + output hidden states 2019-11-26 13:08:12 -05:00
Lysandre
ce9eade29c Initializer range using BertPreTrainedModel 2019-11-26 13:08:12 -05:00
Lysandre
5680a11063 Activation function managed from the config file 2019-11-26 13:08:12 -05:00
Lysandre
1e5b31c388 Several fixes and improvements 2019-11-26 13:08:12 -05:00
Lysandre
e3ea5d1d8d Docstrings 2019-11-26 13:08:12 -05:00
Lysandre
fedac786d4 Tokenization + small fixes 2019-11-26 13:08:12 -05:00
Lysandre
67b422662c Documentation + improved AlbertForMaskedLM 2019-11-26 13:08:12 -05:00
Lysandre
1b92564330 Reorganize and cleanup 2019-11-26 13:08:12 -05:00
Lysandre
12290c0d5c Handles multi layer and multi groups 2019-11-26 13:08:12 -05:00
Lysandre
139affaa8d Albert layer/layer groups 2019-11-26 13:08:12 -05:00
Lysandre
91ccbae788 Accepts multiple sizes 2019-11-26 13:08:12 -05:00
Lysandre
c0c2088333 ALBERT model 2019-11-26 13:08:12 -05:00