diff --git a/README.md b/README.md index 213f61c72b..57bb38048d 100644 --- a/README.md +++ b/README.md @@ -161,31 +161,31 @@ If you'd like to play with the examples, you must [install the library from sour 1. **[ALBERT](https://huggingface.co/transformers/model_doc/albert.html)** (from Google Research and the Toyota Technological Institute at Chicago) released with the paper [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://arxiv.org/abs/1909.11942), by Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, Radu Soricut. 1. **[BART](https://huggingface.co/transformers/model_doc/bart.html)** (from Facebook) released with the paper [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://arxiv.org/pdf/1910.13461.pdf) by Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Ves Stoyanov and Luke Zettlemoyer. 1. **[BERT](https://huggingface.co/transformers/model_doc/bert.html)** (from Google) released with the paper [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://arxiv.org/abs/1810.04805) by Jacob Devlin, Ming-Wei Chang, Kenton Lee and Kristina Toutanova. -1. **[BERT For Sequence Generation](https://tfhub.dev/s?module-type=text-generation&subtype=module,placeholder)** (from Google) released with the paper [Leveraging Pre-trained Checkpoints for Sequence Generation Tasks](https://arxiv.org/abs/1907.12461) by Sascha Rothe, Shashi Narayan, Aliaksei Severyn. +1. **[BERT For Sequence Generation](https://huggingface.co/transformers/model_doc/bertgeneration.html)** (from Google) released with the paper [Leveraging Pre-trained Checkpoints for Sequence Generation Tasks](https://arxiv.org/abs/1907.12461) by Sascha Rothe, Shashi Narayan, Aliaksei Severyn. +1. **[Blenderbot](https://huggingface.co/transformers/master/model_doc/blenderbot.html)** (from Facebook) released with the paper [Recipes for building an open-domain chatbot](https://arxiv.org/abs/2004.13637) by Stephen Roller, Emily Dinan, Naman Goyal, Da Ju, Mary Williamson, Yinhan Liu, Jing Xu, Myle Ott, Kurt Shuster, Eric M. Smith, Y-Lan Boureau, Jason Weston. 1. **[CamemBERT](https://huggingface.co/transformers/model_doc/camembert.html)** (from Inria/Facebook/Sorbonne) released with the paper [CamemBERT: a Tasty French Language Model](https://arxiv.org/abs/1911.03894) by Louis Martin*, Benjamin Muller*, Pedro Javier Ortiz Suárez*, Yoann Dupont, Laurent Romary, Éric Villemonte de la Clergerie, Djamé Seddah and Benoît Sagot. 1. **[CTRL](https://huggingface.co/transformers/model_doc/ctrl.html)** (from Salesforce) released with the paper [CTRL: A Conditional Transformer Language Model for Controllable Generation](https://arxiv.org/abs/1909.05858) by Nitish Shirish Keskar*, Bryan McCann*, Lav R. Varshney, Caiming Xiong and Richard Socher. -1. **[DeBERTa](https://huggingface.co/transformers/model_doc/deberta.html)** (from Microsoft Research) released with the paper [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://arxiv.org/abs/2006.03654) by Pengcheng He, Xiaodong Liu, Jianfeng Gao, Weizhu Chen. +1. **[DeBERTa](https://huggingface.co/transformers/master/model_doc/deberta.html)** (from Microsoft Research) released with the paper [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://arxiv.org/abs/2006.03654) by Pengcheng He, Xiaodong Liu, Jianfeng Gao, Weizhu Chen. 1. **[DialoGPT](https://huggingface.co/transformers/model_doc/dialogpt.html)** (from Microsoft Research) released with the paper [DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation](https://arxiv.org/abs/1911.00536) by Yizhe Zhang, Siqi Sun, Michel Galley, Yen-Chun Chen, Chris Brockett, Xiang Gao, Jianfeng Gao, Jingjing Liu, Bill Dolan. 1. **[DistilBERT](https://huggingface.co/transformers/model_doc/distilbert.html)** (from HuggingFace), released together with the paper [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://arxiv.org/abs/1910.01108) by Victor Sanh, Lysandre Debut and Thomas Wolf. The same method has been applied to compress GPT2 into [DistilGPT2](https://github.com/huggingface/transformers/tree/master/examples/distillation), RoBERTa into [DistilRoBERTa](https://github.com/huggingface/transformers/tree/master/examples/distillation), Multilingual BERT into [DistilmBERT](https://github.com/huggingface/transformers/tree/master/examples/distillation) and a German version of DistilBERT. -1. **[DPR](https://github.com/facebookresearch/DPR)** (from Facebook) released with the paper [Dense Passage Retrieval +1. **[DPR](https://huggingface.co/transformers/model_doc/dpr.html)** (from Facebook) released with the paper [Dense Passage Retrieval for Open-Domain Question Answering](https://arxiv.org/abs/2004.04906) by Vladimir Karpukhin, Barlas Oğuz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 1. **[ELECTRA](https://huggingface.co/transformers/model_doc/electra.html)** (from Google Research/Stanford University) released with the paper [ELECTRA: Pre-training text encoders as discriminators rather than generators](https://arxiv.org/abs/2003.10555) by Kevin Clark, Minh-Thang Luong, Quoc V. Le, Christopher D. Manning. 1. **[FlauBERT](https://huggingface.co/transformers/model_doc/flaubert.html)** (from CNRS) released with the paper [FlauBERT: Unsupervised Language Model Pre-training for French](https://arxiv.org/abs/1912.05372) by Hang Le, Loïc Vial, Jibril Frej, Vincent Segonne, Maximin Coavoux, Benjamin Lecouteux, Alexandre Allauzen, Benoît Crabbé, Laurent Besacier, Didier Schwab. -1. **[Funnel Transformer](https://github.com/laiguokun/Funnel-Transformer)** (from CMU/Google Brain) released with the paper [Funnel-Transformer: Filtering out Sequential Redundancy for Efficient Language Processing](https://arxiv.org/abs/2006.03236) by Zihang Dai, Guokun Lai, Yiming Yang, Quoc V. Le. +1. **[Funnel Transformer](https://huggingface.co/transformers/model_doc/funnel.html)** (from CMU/Google Brain) released with the paper [Funnel-Transformer: Filtering out Sequential Redundancy for Efficient Language Processing](https://arxiv.org/abs/2006.03236) by Zihang Dai, Guokun Lai, Yiming Yang, Quoc V. Le. 1. **[GPT](https://huggingface.co/transformers/model_doc/gpt.html)** (from OpenAI) released with the paper [Improving Language Understanding by Generative Pre-Training](https://blog.openai.com/language-unsupervised/) by Alec Radford, Karthik Narasimhan, Tim Salimans and Ilya Sutskever. 1. **[GPT-2](https://huggingface.co/transformers/model_doc/gpt2.html)** (from OpenAI) released with the paper [Language Models are Unsupervised Multitask Learners](https://blog.openai.com/better-language-models/) by Alec Radford*, Jeffrey Wu*, Rewon Child, David Luan, Dario Amodei** and Ilya Sutskever**. -1. **[LayoutLM](https://github.com/microsoft/unilm/tree/master/layoutlm)** (from Microsoft Research Asia) released with the paper [LayoutLM: Pre-training of Text and Layout for Document Image Understanding](https://arxiv.org/abs/1912.13318) by Yiheng Xu, Minghao Li, Lei Cui, Shaohan Huang, Furu Wei, Ming Zhou. +1. **[LayoutLM](https://huggingface.co/transformers/model_doc/layoutlm.html)** (from Microsoft Research Asia) released with the paper [LayoutLM: Pre-training of Text and Layout for Document Image Understanding](https://arxiv.org/abs/1912.13318) by Yiheng Xu, Minghao Li, Lei Cui, Shaohan Huang, Furu Wei, Ming Zhou. 1. **[Longformer](https://huggingface.co/transformers/model_doc/longformer.html)** (from AllenAI) released with the paper [Longformer: The Long-Document Transformer](https://arxiv.org/abs/2004.05150) by Iz Beltagy, Matthew E. Peters, Arman Cohan. -1. **[LXMERT](https://github.com/airsplay/lxmert)** (from UNC Chapel Hill) released with the paper [LXMERT: Learning Cross-Modality Encoder Representations from Transformers for Open-Domain Question Answering](https://arxiv.org/abs/1908.07490) by Hao Tan and Mohit Bansal. +1. **[LXMERT](https://huggingface.co/transformers/model_doc/lxmert.html)** (from UNC Chapel Hill) released with the paper [LXMERT: Learning Cross-Modality Encoder Representations from Transformers for Open-Domain Question Answering](https://arxiv.org/abs/1908.07490) by Hao Tan and Mohit Bansal. 1. **[MarianMT](https://huggingface.co/transformers/model_doc/marian.html)** Machine translation models trained using [OPUS](http://opus.nlpl.eu/) data by Jörg Tiedemann. The [Marian Framework](https://marian-nmt.github.io/) is being developed by the Microsoft Translator Team. -1. **[MBart](https://github.com/pytorch/fairseq/tree/master/examples/mbart)** (from Facebook) released with the paper [Multilingual Denoising Pre-training for Neural Machine Translation](https://arxiv.org/abs/2001.08210) by Yinhan Liu, Jiatao Gu, Naman Goyal, Xian Li, Sergey Edunov, Marjan Ghazvininejad, Mike Lewis, Luke Zettlemoyer. -1. **[MMBT](https://github.com/facebookresearch/mmbt/)** (from Facebook), released together with the paper a [Supervised Multimodal Bitransformers for Classifying Images and Text](https://arxiv.org/pdf/1909.02950.pdf) by Douwe Kiela, Suvrat Bhooshan, Hamed Firooz, Davide Testuggine. -1. **[Pegasus](https://github.com/google-research/pegasus)** (from Google) released with the paper [PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization](https://arxiv.org/abs/1912.08777)> by Jingqing Zhang, Yao Zhao, Mohammad Saleh and Peter J. Liu. +1. **[MBart](https://huggingface.co/transformers/model_doc/mbart.html)** (from Facebook) released with the paper [Multilingual Denoising Pre-training for Neural Machine Translation](https://arxiv.org/abs/2001.08210) by Yinhan Liu, Jiatao Gu, Naman Goyal, Xian Li, Sergey Edunov, Marjan Ghazvininejad, Mike Lewis, Luke Zettlemoyer. +1. **[Pegasus](https://huggingface.co/transformers/model_doc/pegasus.html)** (from Google) released with the paper [PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization](https://arxiv.org/abs/1912.08777)> by Jingqing Zhang, Yao Zhao, Mohammad Saleh and Peter J. Liu. 1. **[Reformer](https://huggingface.co/transformers/model_doc/reformer.html)** (from Google Research) released with the paper [Reformer: The Efficient Transformer](https://arxiv.org/abs/2001.04451) by Nikita Kitaev, Łukasz Kaiser, Anselm Levskaya. 1. **[RoBERTa](https://huggingface.co/transformers/model_doc/roberta.html)** (from Facebook), released together with the paper a [Robustly Optimized BERT Pretraining Approach](https://arxiv.org/abs/1907.11692) by Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, Veselin Stoyanov. ultilingual BERT into [DistilmBERT](https://github.com/huggingface/transformers/tree/master/examples/distillation) and a German version of DistilBERT. -1. **[SqueezeBert](https://huggingface.co/transformers/model_doc/squeezebert.html)** released with the paper [SqueezeBERT: What can computer vision teach NLP about efficient neural networks?](https://arxiv.org/abs/2006.11316) by Forrest N. Iandola, Albert E. Shaw, Ravi Krishna, and Kurt W. Keutzer. +1. **[SqueezeBert](https://huggingface.co/transformers/master/model_doc/squeezebert.html)** released with the paper [SqueezeBERT: What can computer vision teach NLP about efficient neural networks?](https://arxiv.org/abs/2006.11316) by Forrest N. Iandola, Albert E. Shaw, Ravi Krishna, and Kurt W. Keutzer. 1. **[T5](https://huggingface.co/transformers/model_doc/t5.html)** (from Google AI) released with the paper [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://arxiv.org/abs/1910.10683) by Colin Raffel and Noam Shazeer and Adam Roberts and Katherine Lee and Sharan Narang and Michael Matena and Yanqi Zhou and Wei Li and Peter J. Liu. 1. **[Transformer-XL](https://huggingface.co/transformers/model_doc/transformerxl.html)** (from Google/CMU) released with the paper [Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context](https://arxiv.org/abs/1901.02860) by Zihang Dai*, Zhilin Yang*, Yiming Yang, Jaime Carbonell, Quoc V. Le, Ruslan Salakhutdinov. 1. **[XLM](https://huggingface.co/transformers/model_doc/xlm.html)** (from Facebook) released together with the paper [Cross-lingual Language Model Pretraining](https://arxiv.org/abs/1901.07291) by Guillaume Lample and Alexis Conneau. diff --git a/docs/source/index.rst b/docs/source/index.rst index 3abd21df2a..2c23377419 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -57,112 +57,103 @@ conversion utilities for the following models: .. This list is updated automatically from the README with `make fix-copies`. Do not update manually! -1. `ALBERT `__ (from Google Research and the Toyota - Technological Institute at Chicago) released with the paper `ALBERT: A Lite BERT for Self-supervised Learning of - Language Representations `__, by Zhenzhong Lan, Mingda Chen, Sebastian Goodman, - Kevin Gimpel, Piyush Sharma, Radu Soricut. -2. `BART `__ (from Facebook) released with the paper `BART: - Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension +1. :doc:`ALBERT ` (from Google Research and the Toyota Technological Institute at Chicago) released + with the paper `ALBERT: A Lite BERT for Self-supervised Learning of Language Representations + `__, by Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush + Sharma, Radu Soricut. +2. :doc:`BART ` (from Facebook) released with the paper `BART: Denoising Sequence-to-Sequence + Pre-training for Natural Language Generation, Translation, and Comprehension `__ by Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Ves Stoyanov and Luke Zettlemoyer. -3. `BERT `__ (from Google) released with the paper `BERT: - Pre-training of Deep Bidirectional Transformers for Language Understanding `__ by - Jacob Devlin, Ming-Wei Chang, Kenton Lee and Kristina Toutanova. -4. `BERT For Sequence Generation `__ (from - Google) released with the paper `Leveraging Pre-trained Checkpoints for Sequence Generation Tasks - `__ by Sascha Rothe, Shashi Narayan, Aliaksei Severyn. -5. `CamemBERT `__ (from Inria/Facebook/Sorbonne) released - with the paper `CamemBERT: a Tasty French Language Model `__ by Louis Martin*, - Benjamin Muller*, Pedro Javier Ortiz Suárez*, Yoann Dupont, Laurent Romary, Éric Villemonte de la Clergerie, Djamé - Seddah and Benoît Sagot. -6. `CTRL `__ (from Salesforce) released with the paper `CTRL: - A Conditional Transformer Language Model for Controllable Generation `__ by Nitish - Shirish Keskar*, Bryan McCann*, Lav R. Varshney, Caiming Xiong and Richard Socher. -7. `DeBERTa `__ (from Microsoft Research) released with the - paper `DeBERTa: Decoding-enhanced BERT with Disentangled Attention `__ by +3. :doc:`BERT ` (from Google) released with the paper `BERT: Pre-training of Deep Bidirectional + Transformers for Language Understanding `__ by Jacob Devlin, Ming-Wei Chang, + Kenton Lee and Kristina Toutanova. +4. :doc:`BERT For Sequence Generation ` (from Google) released with the paper `Leveraging + Pre-trained Checkpoints for Sequence Generation Tasks `__ by Sascha Rothe, Shashi + Narayan, Aliaksei Severyn. +5. `Blenderbot `__ (from Facebook) released with + the paper `Recipes for building an open-domain chatbot `__ by Stephen Roller, + Emily Dinan, Naman Goyal, Da Ju, Mary Williamson, Yinhan Liu, Jing Xu, Myle Ott, Kurt Shuster, Eric M. Smith, Y-Lan + Boureau, Jason Weston. +6. :doc:`CamemBERT ` (from Inria/Facebook/Sorbonne) released with the paper `CamemBERT: a Tasty + French Language Model `__ by Louis Martin*, Benjamin Muller*, Pedro Javier Ortiz + Suárez*, Yoann Dupont, Laurent Romary, Éric Villemonte de la Clergerie, Djamé Seddah and Benoît Sagot. +7. :doc:`CTRL ` (from Salesforce) released with the paper `CTRL: A Conditional Transformer Language + Model for Controllable Generation `__ by Nitish Shirish Keskar*, Bryan McCann*, + Lav R. Varshney, Caiming Xiong and Richard Socher. +8. `DeBERTa `__ (from Microsoft Research) released + with the paper `DeBERTa: Decoding-enhanced BERT with Disentangled Attention `__ by Pengcheng He, Xiaodong Liu, Jianfeng Gao, Weizhu Chen. -8. `DialoGPT `__ (from Microsoft Research) released with - the paper `DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation - `__ by Yizhe Zhang, Siqi Sun, Michel Galley, Yen-Chun Chen, Chris Brockett, Xiang - Gao, Jianfeng Gao, Jingjing Liu, Bill Dolan. -9. `DistilBERT `__ (from HuggingFace), released together - with the paper `DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter - `__ by Victor Sanh, Lysandre Debut and Thomas Wolf. The same method has been - applied to compress GPT2 into `DistilGPT2 - `__, RoBERTa into `DistilRoBERTa - `__, Multilingual BERT into - `DistilmBERT `__ and a German version - of DistilBERT. -10. `DPR `__ (from Facebook) released with the paper `Dense Passage Retrieval - for Open-Domain Question Answering `__ by Vladimir Karpukhin, Barlas Oğuz, Sewon - Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. -11. `ELECTRA `__ (from Google Research/Stanford University) - released with the paper `ELECTRA: Pre-training text encoders as discriminators rather than generators - `__ by Kevin Clark, Minh-Thang Luong, Quoc V. Le, Christopher D. Manning. -12. `FlauBERT `__ (from CNRS) released with the paper - `FlauBERT: Unsupervised Language Model Pre-training for French `__ by Hang Le, - Loïc Vial, Jibril Frej, Vincent Segonne, Maximin Coavoux, Benjamin Lecouteux, Alexandre Allauzen, Benoît Crabbé, - Laurent Besacier, Didier Schwab. -13. `Funnel Transformer `__ (from CMU/Google Brain) released with the - paper `Funnel-Transformer: Filtering out Sequential Redundancy for Efficient Language Processing - `__ by Zihang Dai, Guokun Lai, Yiming Yang, Quoc V. Le. -14. `GPT `__ (from OpenAI) released with the paper `Improving - Language Understanding by Generative Pre-Training `__ by Alec - Radford, Karthik Narasimhan, Tim Salimans and Ilya Sutskever. -15. `GPT-2 `__ (from OpenAI) released with the paper `Language - Models are Unsupervised Multitask Learners `__ by Alec Radford*, - Jeffrey Wu*, Rewon Child, David Luan, Dario Amodei** and Ilya Sutskever**. -16. `LayoutLM `__ (from Microsoft Research Asia) released with - the paper `LayoutLM: Pre-training of Text and Layout for Document Image Understanding - `__ by Yiheng Xu, Minghao Li, Lei Cui, Shaohan Huang, Furu Wei, Ming Zhou. -17. `Longformer `__ (from AllenAI) released with the - paper `Longformer: The Long-Document Transformer `__ by Iz Beltagy, Matthew E. - Peters, Arman Cohan. -18. `LXMERT `__ (from UNC Chapel Hill) released with the paper `LXMERT: Learning - Cross-Modality Encoder Representations from Transformers for Open-Domain Question Answering - `__ by Hao Tan and Mohit Bansal. -19. `MarianMT `__ Machine translation models trained using - `OPUS `__ data by Jörg Tiedemann. The `Marian Framework `__ is - being developed by the Microsoft Translator Team. -20. `MBart `__ (from Facebook) released with the paper - `Multilingual Denoising Pre-training for Neural Machine Translation `__ by Yinhan - Liu, Jiatao Gu, Naman Goyal, Xian Li, Sergey Edunov, Marjan Ghazvininejad, Mike Lewis, Luke Zettlemoyer. -21. `MMBT `__ (from Facebook), released together with the paper a - `Supervised Multimodal Bitransformers for Classifying Images and Text `__ by - Douwe Kiela, Suvrat Bhooshan, Hamed Firooz, Davide Testuggine. -22. `Pegasus `__ (from Google) released with the paper `PEGASUS: - Pre-training with Extracted Gap-sentences for Abstractive Summarization `__> by - Jingqing Zhang, Yao Zhao, Mohammad Saleh and Peter J. Liu. -23. `Reformer `__ (from Google Research) released with the - paper `Reformer: The Efficient Transformer `__ by Nikita Kitaev, Łukasz Kaiser, - Anselm Levskaya. -24. `RoBERTa `__ (from Facebook), released together with - the paper a `Robustly Optimized BERT Pretraining Approach `__ by Yinhan Liu, Myle - Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, Veselin Stoyanov. - ultilingual BERT into `DistilmBERT +9. :doc:`DialoGPT ` (from Microsoft Research) released with the paper `DialoGPT: Large-Scale + Generative Pre-training for Conversational Response Generation `__ by Yizhe Zhang, + Siqi Sun, Michel Galley, Yen-Chun Chen, Chris Brockett, Xiang Gao, Jianfeng Gao, Jingjing Liu, Bill Dolan. +10. :doc:`DistilBERT ` (from HuggingFace), released together with the paper `DistilBERT, a + distilled version of BERT: smaller, faster, cheaper and lighter `__ by Victor + Sanh, Lysandre Debut and Thomas Wolf. The same method has been applied to compress GPT2 into `DistilGPT2 + `__, RoBERTa into `DistilRoBERTa + `__, Multilingual BERT into + `DistilmBERT `__ and a German + version of DistilBERT. +11. :doc:`DPR ` (from Facebook) released with the paper `Dense Passage Retrieval for Open-Domain + Question Answering `__ by Vladimir Karpukhin, Barlas Oğuz, Sewon Min, Patrick + Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. +12. :doc:`ELECTRA ` (from Google Research/Stanford University) released with the paper `ELECTRA: + Pre-training text encoders as discriminators rather than generators `__ by Kevin + Clark, Minh-Thang Luong, Quoc V. Le, Christopher D. Manning. +13. :doc:`FlauBERT ` (from CNRS) released with the paper `FlauBERT: Unsupervised Language Model + Pre-training for French `__ by Hang Le, Loïc Vial, Jibril Frej, Vincent Segonne, + Maximin Coavoux, Benjamin Lecouteux, Alexandre Allauzen, Benoît Crabbé, Laurent Besacier, Didier Schwab. +14. :doc:`Funnel Transformer ` (from CMU/Google Brain) released with the paper `Funnel-Transformer: + Filtering out Sequential Redundancy for Efficient Language Processing `__ by + Zihang Dai, Guokun Lai, Yiming Yang, Quoc V. Le. +15. :doc:`GPT ` (from OpenAI) released with the paper `Improving Language Understanding by Generative + Pre-Training `__ by Alec Radford, Karthik Narasimhan, Tim Salimans + and Ilya Sutskever. +16. :doc:`GPT-2 ` (from OpenAI) released with the paper `Language Models are Unsupervised Multitask + Learners `__ by Alec Radford*, Jeffrey Wu*, Rewon Child, David + Luan, Dario Amodei** and Ilya Sutskever**. +17. :doc:`LayoutLM ` (from Microsoft Research Asia) released with the paper `LayoutLM: Pre-training + of Text and Layout for Document Image Understanding `__ by Yiheng Xu, Minghao Li, + Lei Cui, Shaohan Huang, Furu Wei, Ming Zhou. +18. :doc:`Longformer ` (from AllenAI) released with the paper `Longformer: The Long-Document + Transformer `__ by Iz Beltagy, Matthew E. Peters, Arman Cohan. +19. :doc:`LXMERT ` (from UNC Chapel Hill) released with the paper `LXMERT: Learning Cross-Modality + Encoder Representations from Transformers for Open-Domain Question Answering `__ + by Hao Tan and Mohit Bansal. +20. :doc:`MarianMT ` Machine translation models trained using `OPUS `__ data by + Jörg Tiedemann. The `Marian Framework `__ is being developed by the Microsoft + Translator Team. +21. :doc:`MBart ` (from Facebook) released with the paper `Multilingual Denoising Pre-training for + Neural Machine Translation `__ by Yinhan Liu, Jiatao Gu, Naman Goyal, Xian Li, + Sergey Edunov, Marjan Ghazvininejad, Mike Lewis, Luke Zettlemoyer. +22. :doc:`Pegasus ` (from Google) released with the paper `PEGASUS: Pre-training with Extracted + Gap-sentences for Abstractive Summarization `__> by Jingqing Zhang, Yao Zhao, + Mohammad Saleh and Peter J. Liu. +23. :doc:`Reformer ` (from Google Research) released with the paper `Reformer: The Efficient + Transformer `__ by Nikita Kitaev, Łukasz Kaiser, Anselm Levskaya. +24. :doc:`RoBERTa ` (from Facebook), released together with the paper a `Robustly Optimized BERT + Pretraining Approach `__ by Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar + Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, Veselin Stoyanov. ultilingual BERT into `DistilmBERT `__ and a German version of DistilBERT. -25. `SqueezeBert `__ released with the paper +25. `SqueezeBert `__ released with the paper `SqueezeBERT: What can computer vision teach NLP about efficient neural networks? `__ by Forrest N. Iandola, Albert E. Shaw, Ravi Krishna, and Kurt W. Keutzer. -26. `T5 `__ (from Google AI) released with the paper `Exploring - the Limits of Transfer Learning with a Unified Text-to-Text Transformer `__ by - Colin Raffel and Noam Shazeer and Adam Roberts and Katherine Lee and Sharan Narang and Michael Matena and Yanqi - Zhou and Wei Li and Peter J. Liu. -27. `Transformer-XL `__ (from Google/CMU) released - with the paper `Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context - `__ by Zihang Dai*, Zhilin Yang*, Yiming Yang, Jaime Carbonell, Quoc V. Le, - Ruslan Salakhutdinov. -28. `XLM `__ (from Facebook) released together with the paper - `Cross-lingual Language Model Pretraining `__ by Guillaume Lample and Alexis - Conneau. -29. `XLM-RoBERTa `__ (from Facebook AI), released - together with the paper `Unsupervised Cross-lingual Representation Learning at Scale - `__ by Alexis Conneau*, Kartikay Khandelwal*, Naman Goyal, Vishrav Chaudhary, - Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer and Veselin Stoyanov. -30. `XLNet `__ (from Google/CMU) released with the paper - `​XLNet: Generalized Autoregressive Pretraining for Language Understanding `__ by - Zhilin Yang*, Zihang Dai*, Yiming Yang, Jaime Carbonell, Ruslan Salakhutdinov, Quoc V. Le. +26. :doc:`T5 ` (from Google AI) released with the paper `Exploring the Limits of Transfer Learning with a + Unified Text-to-Text Transformer `__ by Colin Raffel and Noam Shazeer and Adam + Roberts and Katherine Lee and Sharan Narang and Michael Matena and Yanqi Zhou and Wei Li and Peter J. Liu. +27. :doc:`Transformer-XL ` (from Google/CMU) released with the paper `Transformer-XL: + Attentive Language Models Beyond a Fixed-Length Context `__ by Zihang Dai*, + Zhilin Yang*, Yiming Yang, Jaime Carbonell, Quoc V. Le, Ruslan Salakhutdinov. +28. :doc:`XLM ` (from Facebook) released together with the paper `Cross-lingual Language Model + Pretraining `__ by Guillaume Lample and Alexis Conneau. +29. :doc:`XLM-RoBERTa ` (from Facebook AI), released together with the paper `Unsupervised + Cross-lingual Representation Learning at Scale `__ by Alexis Conneau*, Kartikay + Khandelwal*, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke + Zettlemoyer and Veselin Stoyanov. +30. :doc:`XLNet ` (from Google/CMU) released with the paper `​XLNet: Generalized Autoregressive + Pretraining for Language Understanding `__ by Zhilin Yang*, Zihang Dai*, Yiming + Yang, Jaime Carbonell, Ruslan Salakhutdinov, Quoc V. Le. 31. `Other community models `__, contributed by the `community `__. diff --git a/setup.py b/setup.py index 38cb92861a..0363c23e24 100644 --- a/setup.py +++ b/setup.py @@ -3,7 +3,9 @@ Simple check list from AllenNLP repo: https://github.com/allenai/allennlp/blob/m To create the package for pypi. -1. Change the version in __init__.py, setup.py as well as docs/source/conf.py. +1. Change the version in __init__.py, setup.py as well as docs/source/conf.py. Remove the master from the links in + the new models of the README: + (https://huggingface.co/transformers/master/model_doc/ -> https://huggingface.co/transformers/model_doc/) 2. Unpin specific versions from setup.py that use a git install. diff --git a/utils/check_copies.py b/utils/check_copies.py index 1a7fcfe04b..264c914e0b 100644 --- a/utils/check_copies.py +++ b/utils/check_copies.py @@ -24,6 +24,7 @@ import tempfile # python utils/check_copies.py TRANSFORMERS_PATH = "src/transformers" PATH_TO_DOCS = "docs/source" +REPO_PATH = "." def find_code_in_transformers(object_name): @@ -175,7 +176,7 @@ def get_model_list(): # If the introduction or the conclusion of the list change, the prompts may need to be updated. _start_prompt = "🤗 Transformers currently provides the following architectures" _end_prompt = "1. Want to contribute a new model?" - with open(os.path.join("README.md"), "r", encoding="utf-8") as f: + with open(os.path.join(REPO_PATH, "README.md"), "r", encoding="utf-8") as f: lines = f.readlines() # Find the start of the list. start_index = 0 @@ -219,7 +220,17 @@ def split_long_line_with_indent(line, max_per_line, indent): def convert_to_rst(model_list, max_per_line=None): """ Convert `model_list` to rst format. """ # Convert **[description](link)** to `description `__ - model_list = re.sub(r"\*\*\[([^\]]*)\]\(([^\)]*)\)\*\*", r"`\1 <\2>`__", model_list) + def _rep_link(match): + title, link = match.groups() + # Keep hard links for the models not released yet + if "master" in link or not link.startswith("https://huggingface.co/transformers"): + return f"`{title} <{link}>`__" + # Convert links to relative links otherwise + else: + link = link[len("https://huggingface.co/transformers/") : -len(".html")] + return f":doc:`{title} <{link}>`" + + model_list = re.sub(r"\*\*\[([^\]]*)\]\(([^\)]*)\)\*\*", _rep_link, model_list) # Convert [description](link) to `description `__ model_list = re.sub(r"\[([^\]]*)\]\(([^\)]*)\)", r"`\1 <\2>`__", model_list)