Commit Graph

13 Commits

Author SHA1 Message Date
keskarnitish
dbed1c5d94 Adding CTRL (squashed commit)
adding conversion script

adding first draft of modeling & tokenization

adding placeholder for test files

bunch of changes

registering the tokenizer/model/etc

tests

change link; something is very VERY wrong here

weird end-of-word thingy going on

i think the tokenization works now ; wrote the unit tests

overall structure works;load w next

the monster is alive!

works after some cleanup as well

adding emacs autosave to gitignore

currently only supporting the 48 layer one; seems to infer fine on my macbook

cleanup

fixing some documentation

fixing some documentation

tests passing?

now works on CUDA also

adding greedy?

adding greedy sampling

works well
2019-10-03 22:29:03 -07:00
VictorSanh
2dc8cb8734 fix unknown imports (*ForMultipleChoice) in run_multiple_choice 2019-09-29 19:51:01 -04:00
Ikuya Yamada
a6a6d9e638 fix padding_idx of RoBERTa model 2019-09-27 19:03:55 -04:00
Julien Chaumond
d8b641c839 6 -> 8 models 2019-09-27 17:22:01 -04:00
Julien Chaumond
c6acbdd50a Close #1304 2019-09-27 17:02:53 -04:00
Agrin Hilmkil
795b3e76ff Add docstring for processor method 2019-09-27 17:32:28 +02:00
Agrin Hilmkil
e31a472801 Fix tensorflow_dataset glue support
`glue_convert_examples_to_features` assumed that tensorflow_dataset
examples contains the features `'sentence1'` and `'sentence2'`. This
commit encapsulates the choice of features in the glue processor and
uses that to parse examples.
2019-09-27 17:16:02 +02:00
LysandreJik
ecfddc6034 Update RoBERTa and GPT-2 Tokenizer documentation (fix #1343) 2019-09-26 16:49:03 -04:00
LysandreJik
36f592cc82 Updated doc for InputExample and InputFeatures 2019-09-26 07:45:40 -04:00
LysandreJik
ad4a393e2e Changed processor documentation architecture. Added documentation for GLUE 2019-09-26 07:45:40 -04:00
thomwolf
80bf868a26 Merge branch 'master' into tf2 2019-09-26 12:04:47 +02:00
thomwolf
481d9c4fb5 Merge branch 'master' into tf2 2019-09-26 12:02:54 +02:00
thomwolf
31c23bd5ee [BIG] pytorch-transformers => transformers 2019-09-26 10:15:53 +02:00