d5b8fe3b90
* Validation split added: custom data files Validation split added in case of no validation file and loading custom data * Updated documentation with custom file usage Updated documentation with custom file usage * Update README.md * Update README.md * Update README.md * Made some suggested stylistic changes * Used logger instead of print. Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com> * Made similar changes to add validation split In case of a missing validation file, a validation split will be used now. * max_train_samples to be used for training only max_train_samples got misplaced, now corrected so that it is applied on training data only, not whole data. * styled * changed ordering * Improved language of documentation Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com> * Improved language of documentation Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com> * Fixed styling issue * Update run_mlm.py Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Examples
This folder contains actively maintained examples of use of 🤗 Transformers using the TensorFlow backend, organized along NLP tasks. It is under construction so we thank you for your patience!
Files containing tf such as run_tf_glue.py are the old-style files that will be rewritten very soon! Files without this such as run_text_classification.py are the newer ones. This message will be removed when the revamp is complete.
The Big Table of Tasks
Here is the list of all our examples:
- with information on whether they are built on top of
Keras(if not, they still work, they might just lack some features), - whether or not they leverage the 🤗 Datasets library.
- links to Colab notebooks to walk through the scripts and run them easily,
| Task | Example datasets | Keras support | 🤗 Datasets | Colab |
|---|---|---|---|---|
language-modeling |
WikiText-2 | - | - | - |
multiple-choice |
SWAG | - | - | - |
question-answering |
SQuAD | - | - | - |
summarization |
XSum | - | - | - |
text-classification |
GLUE | - | - | - |
text-generation |
n/a | - | n/a | - |
token-classification |
CoNLL NER | - | - | - |
translation |
WMT | - | - | - |