LysandreJik
151e4ab4e7
Fix CTRL past
2019-11-05 16:26:51 +00:00
thomwolf
f1e4db2aa8
Fix #1686
2019-11-05 09:38:00 +01:00
thomwolf
b340a910ed
fix tests - flagged as slow all the tests downloading from AWS
2019-11-04 16:03:36 +01:00
thomwolf
f02805da6f
fix tests
2019-11-04 15:42:23 +01:00
thomwolf
1724cee8c4
switch from properties to methods
2019-11-04 15:34:10 +01:00
thomwolf
9b45d0f878
Add common properties input_embeddings and output_embeddings
2019-11-04 12:28:56 +01:00
cregouby
ac29353abe
Fix https://github.com/huggingface/transformers/issues/1673
2019-10-31 10:04:40 +01:00
Thomas Wolf
22838f19fd
Merge pull request #1668 from tlkh/fix-tf-xlm
...
Fixed training for TF XLM
2019-10-30 17:08:00 +01:00
Thomas Wolf
04c69db399
Merge pull request #1628 from huggingface/tfglue
...
run_tf_glue works with all tasks
2019-10-30 17:04:03 +01:00
Thomas Wolf
3df4367244
Merge pull request #1601 from huggingface/clean-roberta
...
Clean roberta model & all tokenizers now add special tokens by default (breaking change)
2019-10-30 17:00:40 +01:00
Thomas Wolf
36174696cc
Merge branch 'master' into clean-roberta
2019-10-30 16:51:06 +01:00
Thomas Wolf
228cdd6a6e
Merge branch 'master' into conditional-generation
2019-10-30 16:40:35 +01:00
Rémi Louf
3cf2020c6b
change kwargs processing
2019-10-30 16:27:51 +01:00
Rémi Louf
a88a0e4413
add tests to encoder-decoder model
2019-10-30 16:06:29 +01:00
Rémi Louf
3f07cd419c
update test on Bert to include decoder mode
2019-10-30 15:09:53 +01:00
Rémi Louf
3b0d2fa30e
rename seq2seq to encoder_decoder
2019-10-30 10:54:46 +01:00
Rémi Louf
9c1bdb5b61
revert renaming of lm_labels to ltr_lm_labels
2019-10-30 10:43:13 +01:00
Timothy Liu
842f3bf049
Fixed training for TF XLM
2019-10-30 01:32:15 +00:00
Rémi Louf
098a89f312
update docstrings; rename lm_labels to more explicit ltr_lm_labels
2019-10-29 20:08:03 +01:00
Rémi Louf
dfce409691
resolve PR comments
2019-10-29 17:10:20 +01:00
Rémi Louf
4c3ac4a7d8
here's one big commit
2019-10-28 10:49:50 +01:00
Rémi Louf
cb26b035c6
remove potential UndefinedError
2019-10-28 10:49:49 +01:00
Rémi Louf
dc580dd4c7
add lm_labels for the LM cross-entropy
2019-10-28 10:49:49 +01:00
Rémi Louf
f873a3edb2
the decoder attends to the output of the encoder stack (last layer)
2019-10-28 10:49:00 +01:00
Lysandre
beaf66b1f3
Remove break
2019-10-24 21:43:28 +00:00
Lysandre
bab6ad01aa
run_tf_glue works with all tasks
2019-10-24 21:41:45 +00:00
Matt Maybeno
b92d68421d
Use roberta model and update doc strings
2019-10-24 14:32:48 -04:00
Matt Maybeno
66085a1321
RoBERTa token classification
...
[WIP] copy paste bert token classification for roberta
2019-10-24 14:32:48 -04:00
Julien Chaumond
ef1b8b2ae5
[CTRL] warn if generation prompt does not start with a control code
...
see also https://github.com/salesforce/ctrl/pull/50
2019-10-22 21:30:32 +00:00
Lysandre
7d709e55ed
Remove
2019-10-22 14:12:33 -04:00
Lysandre
44286b94d3
RoBERTa doesn't print a warning when no special tokens are passed.
2019-10-22 13:46:48 -04:00
Lysandre
777faa8ae7
Fix #1597
2019-10-22 11:26:42 -04:00
Ralph Tang
a2c8c8ef00
Fix hanging when loading pretrained models
...
- Fix hanging when loading pretrained models from the cache without having internet access. This is a widespread issue on supercomputers whose internal compute nodes are firewalled.
2019-10-19 16:19:20 -04:00
VictorSanh
fd97761c5a
soft launch distilroberta
2019-10-17 15:28:58 -04:00
thomwolf
56e2ee4ead
fix model2model
2019-10-17 16:33:31 +02:00
Rémi Louf
bfb9b540d4
add Model2Model to __init__
2019-10-17 12:59:51 +02:00
Rémi Louf
87d60b6e19
reword explanation of encoder_attention_mask
2019-10-17 10:18:19 +02:00
Rémi Louf
638fe7f5a4
correct composition of padding and causal masks
2019-10-17 10:13:07 +02:00
Rémi Louf
4e0f24348f
document the MLM modification + raise exception on MLM training with encoder-decoder
2019-10-17 09:41:53 +02:00
Rémi Louf
624a5644cc
revert black formatting to conform with lib style
2019-10-17 09:27:56 +02:00
Rémi Louf
9b71fc9a18
tying weights is going to be a clusterfuck
2019-10-16 21:31:38 +02:00
Rémi Louf
95ec1d08be
separate inputs into encoder & decoder inputs
2019-10-16 20:55:42 +02:00
Rémi Louf
a424892fab
correct syntax error: dim() and not dims()
2019-10-16 18:24:32 +02:00
Rémi Louf
33c01368b1
remove Bert2Rnd test
2019-10-16 18:13:05 +02:00
Rémi Louf
0752069617
adapt attention masks for the decoder case
...
The introduction of a decoder introduces 2 changes:
- We need to be able to specify a separate mask in the cross
attention to mask the positions corresponding to padding tokens in the
encoder state.
- The self-attention in the decoder needs to be causal on top of not
attending to padding tokens.
2019-10-16 16:12:22 +02:00
Rémi Louf
c5a94a6100
fix function that defines masks in XLM
...
the definition of `get_masks` would blow with the proper combination of
arguments. It was just a matter of moving a definition outside of a
control structure.
2019-10-16 13:00:32 +02:00
Rémi Louf
488a664151
add is_decoder attribute to PretrainedConfig
...
We currenctly instantiate encoders and decoders for the seq2seq by
passing the `is_decoder` keyword argument to the `from_pretrained`
classmethod. On the other hand, the model class looks for the value
of the `is_decoder` attribute in its config.
In order for the value to propagate from the kwarg to the configuration
we simply need to define `is_decoder` as an attribute to the base
`PretrainedConfig`, with a default at `False`.
2019-10-15 21:03:32 +02:00
Rémi Louf
4c81960b9b
comment the seq2seq functions
2019-10-15 20:52:28 +02:00
Rémi Louf
6d6c326737
take path to pretrained for encoder and decoder for init
2019-10-15 16:08:27 +02:00
Rémi Louf
19e9964780
remove Bert2Bert from module declaration
2019-10-15 15:20:28 +02:00
