From b0f0086fa45f455d2ab05406348017566f954d91 Mon Sep 17 00:00:00 2001 From: Alissa <96190409+alissadb@users.noreply.github.com> Date: Thu, 16 Feb 2023 18:44:28 +0100 Subject: [PATCH] Add OPT resources to the transformers documentation (#21625) * Add resources to OPT * Add additional resources for OPT * Remove -{" "} after * Change bitsnbytes to bitsandbytes * Revert formatting * Revert automatic format changes * Remove - sign after --- docs/source/en/model_doc/opt.mdx | 34 +++++++++++++++++++++++++++++--- 1 file changed, 31 insertions(+), 3 deletions(-) diff --git a/docs/source/en/model_doc/opt.mdx b/docs/source/en/model_doc/opt.mdx index 612689678f..6bf8135217 100644 --- a/docs/source/en/model_doc/opt.mdx +++ b/docs/source/en/model_doc/opt.mdx @@ -15,8 +15,7 @@ specific language governing permissions and limitations under the License. ## Overview The OPT model was proposed in [Open Pre-trained Transformer Language Models](https://arxiv.org/pdf/2205.01068) by Meta AI. -OPT is a series of open-sourced large causal language models which perform similar in performance to GPT3. - +OPT is a series of open-sourced large causal language models which perform similar in performance to GPT3. The abstract from the paper is the following: @@ -29,6 +28,35 @@ Tips: This model was contributed by [Arthur Zucker](https://huggingface.co/ArthurZ), [Younes Belkada](https://huggingface.co/ybelkada), and [Patrick Von Platen](https://huggingface.co/patrickvonplaten). The original code can be found [here](https://github.com/facebookresearch/metaseq). +## Resources + +A list of official Hugging Face and community (indicated by 🌎) resources to help you get started with OPT. If you're +interested in submitting a resource to be included here, please feel free to open a Pull Request and we will review it. +The resource should ideally demonstrate something new instead of duplicating an existing resource. + + + +- A notebook on [fine-tuning OPT with PEFT, bitsandbytes, and Transformers](https://colab.research.google.com/drive/1jCkpikz0J2o20FBQmYmAGdiKmJGOMo-o?usp=sharing). 🌎 +- A blog post on [decoding strategies with OPT](https://huggingface.co/blog/introducing-csearch#62-example-two---opt). +- [Causal language modeling](https://huggingface.co/course/en/chapter7/6?fw=pt#training-a-causal-language-model-from-scratch) chapter of the 🤗 Hugging Face Course. +- [`OPTForCausalLM`] is supported by this [causal language modeling example script](https://github.com/huggingface/transformers/tree/main/examples/pytorch/language-modeling#gpt-2gpt-and-causal-language-modeling) and [notebook](https://colab.research.google.com/github/huggingface/notebooks/blob/main/examples/language_modeling.ipynb). +- [`TFOPTForCausalLM`] is supported by this [causal language modeling example script](https://github.com/huggingface/transformers/tree/main/examples/tensorflow/language-modeling#run_clmpy) and [notebook](https://colab.research.google.com/github/huggingface/notebooks/blob/main/examples/language_modeling-tf.ipynb). +- [`FlaxOPTForCausalLM`] is supported by this [causal language modeling example script](https://github.com/huggingface/transformers/tree/main/examples/flax/language-modeling#causal-language-modeling). + + + +- [Token classification](https://huggingface.co/course/chapter7/2?fw=pt) chapter of the 🤗 Hugging Face Course. +- [`OPTForSequenceClassification`] is supported by this [example script](https://github.com/huggingface/transformers/tree/main/examples/pytorch/text-classification) and [notebook](https://colab.research.google.com/github/huggingface/notebooks/blob/main/examples/text_classification.ipynb). + + + +- [`OPTForQuestionAnswering`] is supported by this [question answering example script](https://github.com/huggingface/transformers/tree/main/examples/pytorch/question-answering) and [notebook](https://colab.research.google.com/github/huggingface/notebooks/blob/main/examples/question_answering.ipynb). +- [Question answering](https://huggingface.co/course/chapter7/7?fw=pt) chapter + of the 🤗 Hugging Face Course. + +⚡️ Inference + +- A blog bost on [How 🤗 Accelerate runs very large models thanks to PyTorch](https://huggingface.co/blog/accelerate-large-models) with OPT. ## OPTConfig @@ -73,4 +101,4 @@ The original code can be found [here](https://github.com/facebookresearch/metase ## FlaxOPTForCausalLM [[autodoc]] FlaxOPTForCausalLM - - __call__ \ No newline at end of file + - __call__