thomwolf
b340a910ed
fix tests - flagged as slow all the tests downloading from AWS
2019-11-04 16:03:36 +01:00
thomwolf
f02805da6f
fix tests
2019-11-04 15:42:23 +01:00
thomwolf
1724cee8c4
switch from properties to methods
2019-11-04 15:34:10 +01:00
thomwolf
9b45d0f878
Add common properties input_embeddings and output_embeddings
2019-11-04 12:28:56 +01:00
Thomas Wolf
3df4367244
Merge pull request #1601 from huggingface/clean-roberta
...
Clean roberta model & all tokenizers now add special tokens by default (breaking change)
2019-10-30 17:00:40 +01:00
Thomas Wolf
36174696cc
Merge branch 'master' into clean-roberta
2019-10-30 16:51:06 +01:00
Thomas Wolf
228cdd6a6e
Merge branch 'master' into conditional-generation
2019-10-30 16:40:35 +01:00
Rémi Louf
a88a0e4413
add tests to encoder-decoder model
2019-10-30 16:06:29 +01:00
Rémi Louf
3f07cd419c
update test on Bert to include decoder mode
2019-10-30 15:09:53 +01:00
Matt Maybeno
66085a1321
RoBERTa token classification
...
[WIP] copy paste bert token classification for roberta
2019-10-24 14:32:48 -04:00
Lysandre
7d709e55ed
Remove
2019-10-22 14:12:33 -04:00
Rémi Louf
33c01368b1
remove Bert2Rnd test
2019-10-16 18:13:05 +02:00
thomwolf
898ce064f8
add tests on TF2.0 & PT checkpoint => model convertion functions
2019-10-15 10:04:19 +02:00
thomwolf
18a3cef7d5
no nans
2019-10-11 16:09:42 +02:00
thomwolf
1f5d9513d8
fix test
2019-10-11 15:55:01 +02:00
thomwolf
0f9fc4fbde
adding option to desactivate past/memory outputs
2019-10-11 15:47:08 +02:00
Rémi Louf
1e68c28670
add test for initialization of Bert2Rnd
2019-10-10 18:07:11 +02:00
thomwolf
da26bae61b
adding more tests on TF and pytorch serialization - updating configuration for better serialization
2019-10-10 14:30:48 +02:00
thomwolf
bb04edb45b
Add tests that TF 2.0 model can be integrated with other Keras modules
2019-10-10 13:08:24 +02:00
Lysandre Debut
2431fea98a
Merge pull request #1383 from keskarnitish/master
...
Adding CTRL
2019-10-09 11:31:05 -04:00
thomwolf
07d055f849
higher tolerance
2019-10-09 17:10:04 +02:00
thomwolf
c19b8e4ae0
fixing CTRL tests and OpenAI GPT tests
2019-10-09 13:51:05 +02:00
thomwolf
6dce6dda1b
fixing TF 2.0 model - adding more severe test on pt/tf equivalence
2019-10-09 11:57:55 +02:00
thomwolf
c56d921dda
adding TF 2.0 model
2019-10-09 11:07:43 +02:00
thomwolf
248314772f
fix tokenization
2019-10-08 17:19:28 +02:00
thomwolf
3edfa1d6aa
update model to use past
2019-10-08 17:11:58 +02:00
Rémi Louf
61ed889005
remove old seq2seq file
2019-10-08 16:30:58 +02:00
Rémi Louf
8abfee9ec3
rename Bert2Bert -> Bert2Rnd
2019-10-08 16:30:58 +02:00
Rémi Louf
82628b0fc9
add a placeholder test
2019-10-08 16:30:58 +02:00
Rémi Louf
4446c02b8a
add wireframe for seq2seq model
2019-10-07 12:04:05 +02:00
thomwolf
78ef1a9930
fixes
2019-10-04 17:59:44 -04:00
thomwolf
6c1d0bc066
update encode_plus - add truncation strategies
2019-10-04 17:38:38 -04:00
keskarnitish
dbed1c5d94
Adding CTRL (squashed commit)
...
adding conversion script
adding first draft of modeling & tokenization
adding placeholder for test files
bunch of changes
registering the tokenizer/model/etc
tests
change link; something is very VERY wrong here
weird end-of-word thingy going on
i think the tokenization works now ; wrote the unit tests
overall structure works;load w next
the monster is alive!
works after some cleanup as well
adding emacs autosave to gitignore
currently only supporting the 48 layer one; seems to infer fine on my macbook
cleanup
fixing some documentation
fixing some documentation
tests passing?
now works on CUDA also
adding greedy?
adding greedy sampling
works well
2019-10-03 22:29:03 -07:00
LysandreJik
aebd83230f
Update naming + remove f string in run_lm_finetuning example
2019-10-03 11:31:36 -04:00
LysandreJik
651bfb7ad5
always_truncate by default
2019-10-03 11:31:36 -04:00
LysandreJik
cc412edd42
Supports already existing special tokens
2019-10-03 11:31:36 -04:00
LysandreJik
2f259b228e
Sequence IDS
2019-10-03 11:31:36 -04:00
LysandreJik
7c789c337d
Always truncate argument in the encode method
2019-10-03 11:31:36 -04:00
thomwolf
31c23bd5ee
[BIG] pytorch-transformers => transformers
2019-09-26 10:15:53 +02:00
