From c4ac7a76db7aa40bd636a59dc71ef1dd27b2c2c4 Mon Sep 17 00:00:00 2001 From: LysandreJik Date: Wed, 25 Sep 2019 07:09:30 -0400 Subject: [PATCH] GLUE processors --- docs/source/index.rst | 1 + docs/source/main_classes/processors.rst | 45 +++++++++++++++++++++++++ 2 files changed, 46 insertions(+) create mode 100644 docs/source/main_classes/processors.rst diff --git a/docs/source/index.rst b/docs/source/index.rst index 3a30b61b2c..0d5bd01352 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -67,6 +67,7 @@ The library currently contains PyTorch and Tensorflow implementations, pre-train main_classes/model main_classes/tokenizer main_classes/optimizer_schedules + main_classes/processors .. toctree:: :maxdepth: 2 diff --git a/docs/source/main_classes/processors.rst b/docs/source/main_classes/processors.rst new file mode 100644 index 0000000000..12e5339ddb --- /dev/null +++ b/docs/source/main_classes/processors.rst @@ -0,0 +1,45 @@ +Processors +---------------------------------------------------- + +This library includes processors for several traditional tasks. These processors can be used to process a dataset into +examples that can be fed to a model. + +``GLUE`` +~~~~~~~~~~~~~~~~~~~~~ + +`General Language Understanding Evaluation (GLUE)`__ is a benchmark that evaluates +the performance of models across a diverse set of existing NLU tasks. It was released together with the paper +`GLUE: A multi-task benchmark and analysis platform for natural language understanding`__ + +This library hosts a total of 10 processors for the following tasks: MRPC, MNLI, MNLI (mismatched), +CoLA, SST2, STSB, QQP, QNLI, RTE and WNLI. + +.. autoclass:: pytorch_transformers.data.processors.glue.MrpcProcessor + :members: + +.. autoclass:: pytorch_transformers.data.processors.glue.MnliProcessor + :members: + +.. autoclass:: pytorch_transformers.data.processors.glue.MnliMismatchedProcessor + :members: + +.. autoclass:: pytorch_transformers.data.processors.glue.ColaProcessor + :members: + +.. autoclass:: pytorch_transformers.data.processors.glue.Sst2Processor + :members: + +.. autoclass:: pytorch_transformers.data.processors.glue.StsbProcessor + :members: + +.. autoclass:: pytorch_transformers.data.processors.glue.QqpProcessor + :members: + +.. autoclass:: pytorch_transformers.data.processors.glue.QnliProcessor + :members: + +.. autoclass:: pytorch_transformers.data.processors.glue.RteProcessor + :members: + +.. autoclass:: pytorch_transformers.data.processors.glue.WnliProcessor + :members: