diff --git a/README.md b/README.md index 57bb38048d..213f61c72b 100644 --- a/README.md +++ b/README.md @@ -161,31 +161,31 @@ If you'd like to play with the examples, you must [install the library from sour 1. **[ALBERT](https://huggingface.co/transformers/model_doc/albert.html)** (from Google Research and the Toyota Technological Institute at Chicago) released with the paper [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://arxiv.org/abs/1909.11942), by Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, Radu Soricut. 1. **[BART](https://huggingface.co/transformers/model_doc/bart.html)** (from Facebook) released with the paper [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://arxiv.org/pdf/1910.13461.pdf) by Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Ves Stoyanov and Luke Zettlemoyer. 1. **[BERT](https://huggingface.co/transformers/model_doc/bert.html)** (from Google) released with the paper [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://arxiv.org/abs/1810.04805) by Jacob Devlin, Ming-Wei Chang, Kenton Lee and Kristina Toutanova. -1. **[BERT For Sequence Generation](https://huggingface.co/transformers/model_doc/bertgeneration.html)** (from Google) released with the paper [Leveraging Pre-trained Checkpoints for Sequence Generation Tasks](https://arxiv.org/abs/1907.12461) by Sascha Rothe, Shashi Narayan, Aliaksei Severyn. -1. **[Blenderbot](https://huggingface.co/transformers/master/model_doc/blenderbot.html)** (from Facebook) released with the paper [Recipes for building an open-domain chatbot](https://arxiv.org/abs/2004.13637) by Stephen Roller, Emily Dinan, Naman Goyal, Da Ju, Mary Williamson, Yinhan Liu, Jing Xu, Myle Ott, Kurt Shuster, Eric M. Smith, Y-Lan Boureau, Jason Weston. +1. **[BERT For Sequence Generation](https://tfhub.dev/s?module-type=text-generation&subtype=module,placeholder)** (from Google) released with the paper [Leveraging Pre-trained Checkpoints for Sequence Generation Tasks](https://arxiv.org/abs/1907.12461) by Sascha Rothe, Shashi Narayan, Aliaksei Severyn. 1. **[CamemBERT](https://huggingface.co/transformers/model_doc/camembert.html)** (from Inria/Facebook/Sorbonne) released with the paper [CamemBERT: a Tasty French Language Model](https://arxiv.org/abs/1911.03894) by Louis Martin*, Benjamin Muller*, Pedro Javier Ortiz Suárez*, Yoann Dupont, Laurent Romary, Éric Villemonte de la Clergerie, Djamé Seddah and Benoît Sagot. 1. **[CTRL](https://huggingface.co/transformers/model_doc/ctrl.html)** (from Salesforce) released with the paper [CTRL: A Conditional Transformer Language Model for Controllable Generation](https://arxiv.org/abs/1909.05858) by Nitish Shirish Keskar*, Bryan McCann*, Lav R. Varshney, Caiming Xiong and Richard Socher. -1. **[DeBERTa](https://huggingface.co/transformers/master/model_doc/deberta.html)** (from Microsoft Research) released with the paper [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://arxiv.org/abs/2006.03654) by Pengcheng He, Xiaodong Liu, Jianfeng Gao, Weizhu Chen. +1. **[DeBERTa](https://huggingface.co/transformers/model_doc/deberta.html)** (from Microsoft Research) released with the paper [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://arxiv.org/abs/2006.03654) by Pengcheng He, Xiaodong Liu, Jianfeng Gao, Weizhu Chen. 1. **[DialoGPT](https://huggingface.co/transformers/model_doc/dialogpt.html)** (from Microsoft Research) released with the paper [DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation](https://arxiv.org/abs/1911.00536) by Yizhe Zhang, Siqi Sun, Michel Galley, Yen-Chun Chen, Chris Brockett, Xiang Gao, Jianfeng Gao, Jingjing Liu, Bill Dolan. 1. **[DistilBERT](https://huggingface.co/transformers/model_doc/distilbert.html)** (from HuggingFace), released together with the paper [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://arxiv.org/abs/1910.01108) by Victor Sanh, Lysandre Debut and Thomas Wolf. The same method has been applied to compress GPT2 into [DistilGPT2](https://github.com/huggingface/transformers/tree/master/examples/distillation), RoBERTa into [DistilRoBERTa](https://github.com/huggingface/transformers/tree/master/examples/distillation), Multilingual BERT into [DistilmBERT](https://github.com/huggingface/transformers/tree/master/examples/distillation) and a German version of DistilBERT. -1. **[DPR](https://huggingface.co/transformers/model_doc/dpr.html)** (from Facebook) released with the paper [Dense Passage Retrieval +1. **[DPR](https://github.com/facebookresearch/DPR)** (from Facebook) released with the paper [Dense Passage Retrieval for Open-Domain Question Answering](https://arxiv.org/abs/2004.04906) by Vladimir Karpukhin, Barlas Oğuz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 1. **[ELECTRA](https://huggingface.co/transformers/model_doc/electra.html)** (from Google Research/Stanford University) released with the paper [ELECTRA: Pre-training text encoders as discriminators rather than generators](https://arxiv.org/abs/2003.10555) by Kevin Clark, Minh-Thang Luong, Quoc V. Le, Christopher D. Manning. 1. **[FlauBERT](https://huggingface.co/transformers/model_doc/flaubert.html)** (from CNRS) released with the paper [FlauBERT: Unsupervised Language Model Pre-training for French](https://arxiv.org/abs/1912.05372) by Hang Le, Loïc Vial, Jibril Frej, Vincent Segonne, Maximin Coavoux, Benjamin Lecouteux, Alexandre Allauzen, Benoît Crabbé, Laurent Besacier, Didier Schwab. -1. **[Funnel Transformer](https://huggingface.co/transformers/model_doc/funnel.html)** (from CMU/Google Brain) released with the paper [Funnel-Transformer: Filtering out Sequential Redundancy for Efficient Language Processing](https://arxiv.org/abs/2006.03236) by Zihang Dai, Guokun Lai, Yiming Yang, Quoc V. Le. +1. **[Funnel Transformer](https://github.com/laiguokun/Funnel-Transformer)** (from CMU/Google Brain) released with the paper [Funnel-Transformer: Filtering out Sequential Redundancy for Efficient Language Processing](https://arxiv.org/abs/2006.03236) by Zihang Dai, Guokun Lai, Yiming Yang, Quoc V. Le. 1. **[GPT](https://huggingface.co/transformers/model_doc/gpt.html)** (from OpenAI) released with the paper [Improving Language Understanding by Generative Pre-Training](https://blog.openai.com/language-unsupervised/) by Alec Radford, Karthik Narasimhan, Tim Salimans and Ilya Sutskever. 1. **[GPT-2](https://huggingface.co/transformers/model_doc/gpt2.html)** (from OpenAI) released with the paper [Language Models are Unsupervised Multitask Learners](https://blog.openai.com/better-language-models/) by Alec Radford*, Jeffrey Wu*, Rewon Child, David Luan, Dario Amodei** and Ilya Sutskever**. -1. **[LayoutLM](https://huggingface.co/transformers/model_doc/layoutlm.html)** (from Microsoft Research Asia) released with the paper [LayoutLM: Pre-training of Text and Layout for Document Image Understanding](https://arxiv.org/abs/1912.13318) by Yiheng Xu, Minghao Li, Lei Cui, Shaohan Huang, Furu Wei, Ming Zhou. +1. **[LayoutLM](https://github.com/microsoft/unilm/tree/master/layoutlm)** (from Microsoft Research Asia) released with the paper [LayoutLM: Pre-training of Text and Layout for Document Image Understanding](https://arxiv.org/abs/1912.13318) by Yiheng Xu, Minghao Li, Lei Cui, Shaohan Huang, Furu Wei, Ming Zhou. 1. **[Longformer](https://huggingface.co/transformers/model_doc/longformer.html)** (from AllenAI) released with the paper [Longformer: The Long-Document Transformer](https://arxiv.org/abs/2004.05150) by Iz Beltagy, Matthew E. Peters, Arman Cohan. -1. **[LXMERT](https://huggingface.co/transformers/model_doc/lxmert.html)** (from UNC Chapel Hill) released with the paper [LXMERT: Learning Cross-Modality Encoder Representations from Transformers for Open-Domain Question Answering](https://arxiv.org/abs/1908.07490) by Hao Tan and Mohit Bansal. +1. **[LXMERT](https://github.com/airsplay/lxmert)** (from UNC Chapel Hill) released with the paper [LXMERT: Learning Cross-Modality Encoder Representations from Transformers for Open-Domain Question Answering](https://arxiv.org/abs/1908.07490) by Hao Tan and Mohit Bansal. 1. **[MarianMT](https://huggingface.co/transformers/model_doc/marian.html)** Machine translation models trained using [OPUS](http://opus.nlpl.eu/) data by Jörg Tiedemann. The [Marian Framework](https://marian-nmt.github.io/) is being developed by the Microsoft Translator Team. -1. **[MBart](https://huggingface.co/transformers/model_doc/mbart.html)** (from Facebook) released with the paper [Multilingual Denoising Pre-training for Neural Machine Translation](https://arxiv.org/abs/2001.08210) by Yinhan Liu, Jiatao Gu, Naman Goyal, Xian Li, Sergey Edunov, Marjan Ghazvininejad, Mike Lewis, Luke Zettlemoyer. -1. **[Pegasus](https://huggingface.co/transformers/model_doc/pegasus.html)** (from Google) released with the paper [PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization](https://arxiv.org/abs/1912.08777)> by Jingqing Zhang, Yao Zhao, Mohammad Saleh and Peter J. Liu. +1. **[MBart](https://github.com/pytorch/fairseq/tree/master/examples/mbart)** (from Facebook) released with the paper [Multilingual Denoising Pre-training for Neural Machine Translation](https://arxiv.org/abs/2001.08210) by Yinhan Liu, Jiatao Gu, Naman Goyal, Xian Li, Sergey Edunov, Marjan Ghazvininejad, Mike Lewis, Luke Zettlemoyer. +1. **[MMBT](https://github.com/facebookresearch/mmbt/)** (from Facebook), released together with the paper a [Supervised Multimodal Bitransformers for Classifying Images and Text](https://arxiv.org/pdf/1909.02950.pdf) by Douwe Kiela, Suvrat Bhooshan, Hamed Firooz, Davide Testuggine. +1. **[Pegasus](https://github.com/google-research/pegasus)** (from Google) released with the paper [PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization](https://arxiv.org/abs/1912.08777)> by Jingqing Zhang, Yao Zhao, Mohammad Saleh and Peter J. Liu. 1. **[Reformer](https://huggingface.co/transformers/model_doc/reformer.html)** (from Google Research) released with the paper [Reformer: The Efficient Transformer](https://arxiv.org/abs/2001.04451) by Nikita Kitaev, Łukasz Kaiser, Anselm Levskaya. 1. **[RoBERTa](https://huggingface.co/transformers/model_doc/roberta.html)** (from Facebook), released together with the paper a [Robustly Optimized BERT Pretraining Approach](https://arxiv.org/abs/1907.11692) by Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, Veselin Stoyanov. ultilingual BERT into [DistilmBERT](https://github.com/huggingface/transformers/tree/master/examples/distillation) and a German version of DistilBERT. -1. **[SqueezeBert](https://huggingface.co/transformers/master/model_doc/squeezebert.html)** released with the paper [SqueezeBERT: What can computer vision teach NLP about efficient neural networks?](https://arxiv.org/abs/2006.11316) by Forrest N. Iandola, Albert E. Shaw, Ravi Krishna, and Kurt W. Keutzer. +1. **[SqueezeBert](https://huggingface.co/transformers/model_doc/squeezebert.html)** released with the paper [SqueezeBERT: What can computer vision teach NLP about efficient neural networks?](https://arxiv.org/abs/2006.11316) by Forrest N. Iandola, Albert E. Shaw, Ravi Krishna, and Kurt W. Keutzer. 1. **[T5](https://huggingface.co/transformers/model_doc/t5.html)** (from Google AI) released with the paper [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://arxiv.org/abs/1910.10683) by Colin Raffel and Noam Shazeer and Adam Roberts and Katherine Lee and Sharan Narang and Michael Matena and Yanqi Zhou and Wei Li and Peter J. Liu. 1. **[Transformer-XL](https://huggingface.co/transformers/model_doc/transformerxl.html)** (from Google/CMU) released with the paper [Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context](https://arxiv.org/abs/1901.02860) by Zihang Dai*, Zhilin Yang*, Yiming Yang, Jaime Carbonell, Quoc V. Le, Ruslan Salakhutdinov. 1. **[XLM](https://huggingface.co/transformers/model_doc/xlm.html)** (from Facebook) released together with the paper [Cross-lingual Language Model Pretraining](https://arxiv.org/abs/1901.07291) by Guillaume Lample and Alexis Conneau. diff --git a/docs/source/index.rst b/docs/source/index.rst index 2c23377419..3abd21df2a 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -57,103 +57,112 @@ conversion utilities for the following models: .. This list is updated automatically from the README with `make fix-copies`. Do not update manually! -1. :doc:`ALBERT ` (from Google Research and the Toyota Technological Institute at Chicago) released - with the paper `ALBERT: A Lite BERT for Self-supervised Learning of Language Representations - `__, by Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush - Sharma, Radu Soricut. -2. :doc:`BART ` (from Facebook) released with the paper `BART: Denoising Sequence-to-Sequence - Pre-training for Natural Language Generation, Translation, and Comprehension +1. `ALBERT `__ (from Google Research and the Toyota + Technological Institute at Chicago) released with the paper `ALBERT: A Lite BERT for Self-supervised Learning of + Language Representations `__, by Zhenzhong Lan, Mingda Chen, Sebastian Goodman, + Kevin Gimpel, Piyush Sharma, Radu Soricut. +2. `BART `__ (from Facebook) released with the paper `BART: + Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension `__ by Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Ves Stoyanov and Luke Zettlemoyer. -3. :doc:`BERT ` (from Google) released with the paper `BERT: Pre-training of Deep Bidirectional - Transformers for Language Understanding `__ by Jacob Devlin, Ming-Wei Chang, - Kenton Lee and Kristina Toutanova. -4. :doc:`BERT For Sequence Generation ` (from Google) released with the paper `Leveraging - Pre-trained Checkpoints for Sequence Generation Tasks `__ by Sascha Rothe, Shashi - Narayan, Aliaksei Severyn. -5. `Blenderbot `__ (from Facebook) released with - the paper `Recipes for building an open-domain chatbot `__ by Stephen Roller, - Emily Dinan, Naman Goyal, Da Ju, Mary Williamson, Yinhan Liu, Jing Xu, Myle Ott, Kurt Shuster, Eric M. Smith, Y-Lan - Boureau, Jason Weston. -6. :doc:`CamemBERT ` (from Inria/Facebook/Sorbonne) released with the paper `CamemBERT: a Tasty - French Language Model `__ by Louis Martin*, Benjamin Muller*, Pedro Javier Ortiz - Suárez*, Yoann Dupont, Laurent Romary, Éric Villemonte de la Clergerie, Djamé Seddah and Benoît Sagot. -7. :doc:`CTRL ` (from Salesforce) released with the paper `CTRL: A Conditional Transformer Language - Model for Controllable Generation `__ by Nitish Shirish Keskar*, Bryan McCann*, - Lav R. Varshney, Caiming Xiong and Richard Socher. -8. `DeBERTa `__ (from Microsoft Research) released - with the paper `DeBERTa: Decoding-enhanced BERT with Disentangled Attention `__ by +3. `BERT `__ (from Google) released with the paper `BERT: + Pre-training of Deep Bidirectional Transformers for Language Understanding `__ by + Jacob Devlin, Ming-Wei Chang, Kenton Lee and Kristina Toutanova. +4. `BERT For Sequence Generation `__ (from + Google) released with the paper `Leveraging Pre-trained Checkpoints for Sequence Generation Tasks + `__ by Sascha Rothe, Shashi Narayan, Aliaksei Severyn. +5. `CamemBERT `__ (from Inria/Facebook/Sorbonne) released + with the paper `CamemBERT: a Tasty French Language Model `__ by Louis Martin*, + Benjamin Muller*, Pedro Javier Ortiz Suárez*, Yoann Dupont, Laurent Romary, Éric Villemonte de la Clergerie, Djamé + Seddah and Benoît Sagot. +6. `CTRL `__ (from Salesforce) released with the paper `CTRL: + A Conditional Transformer Language Model for Controllable Generation `__ by Nitish + Shirish Keskar*, Bryan McCann*, Lav R. Varshney, Caiming Xiong and Richard Socher. +7. `DeBERTa `__ (from Microsoft Research) released with the + paper `DeBERTa: Decoding-enhanced BERT with Disentangled Attention `__ by Pengcheng He, Xiaodong Liu, Jianfeng Gao, Weizhu Chen. -9. :doc:`DialoGPT ` (from Microsoft Research) released with the paper `DialoGPT: Large-Scale - Generative Pre-training for Conversational Response Generation `__ by Yizhe Zhang, - Siqi Sun, Michel Galley, Yen-Chun Chen, Chris Brockett, Xiang Gao, Jianfeng Gao, Jingjing Liu, Bill Dolan. -10. :doc:`DistilBERT ` (from HuggingFace), released together with the paper `DistilBERT, a - distilled version of BERT: smaller, faster, cheaper and lighter `__ by Victor - Sanh, Lysandre Debut and Thomas Wolf. The same method has been applied to compress GPT2 into `DistilGPT2 - `__, RoBERTa into `DistilRoBERTa - `__, Multilingual BERT into - `DistilmBERT `__ and a German - version of DistilBERT. -11. :doc:`DPR ` (from Facebook) released with the paper `Dense Passage Retrieval for Open-Domain - Question Answering `__ by Vladimir Karpukhin, Barlas Oğuz, Sewon Min, Patrick - Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. -12. :doc:`ELECTRA ` (from Google Research/Stanford University) released with the paper `ELECTRA: - Pre-training text encoders as discriminators rather than generators `__ by Kevin - Clark, Minh-Thang Luong, Quoc V. Le, Christopher D. Manning. -13. :doc:`FlauBERT ` (from CNRS) released with the paper `FlauBERT: Unsupervised Language Model - Pre-training for French `__ by Hang Le, Loïc Vial, Jibril Frej, Vincent Segonne, - Maximin Coavoux, Benjamin Lecouteux, Alexandre Allauzen, Benoît Crabbé, Laurent Besacier, Didier Schwab. -14. :doc:`Funnel Transformer ` (from CMU/Google Brain) released with the paper `Funnel-Transformer: - Filtering out Sequential Redundancy for Efficient Language Processing `__ by - Zihang Dai, Guokun Lai, Yiming Yang, Quoc V. Le. -15. :doc:`GPT ` (from OpenAI) released with the paper `Improving Language Understanding by Generative - Pre-Training `__ by Alec Radford, Karthik Narasimhan, Tim Salimans - and Ilya Sutskever. -16. :doc:`GPT-2 ` (from OpenAI) released with the paper `Language Models are Unsupervised Multitask - Learners `__ by Alec Radford*, Jeffrey Wu*, Rewon Child, David - Luan, Dario Amodei** and Ilya Sutskever**. -17. :doc:`LayoutLM ` (from Microsoft Research Asia) released with the paper `LayoutLM: Pre-training - of Text and Layout for Document Image Understanding `__ by Yiheng Xu, Minghao Li, - Lei Cui, Shaohan Huang, Furu Wei, Ming Zhou. -18. :doc:`Longformer ` (from AllenAI) released with the paper `Longformer: The Long-Document - Transformer `__ by Iz Beltagy, Matthew E. Peters, Arman Cohan. -19. :doc:`LXMERT ` (from UNC Chapel Hill) released with the paper `LXMERT: Learning Cross-Modality - Encoder Representations from Transformers for Open-Domain Question Answering `__ - by Hao Tan and Mohit Bansal. -20. :doc:`MarianMT ` Machine translation models trained using `OPUS `__ data by - Jörg Tiedemann. The `Marian Framework `__ is being developed by the Microsoft - Translator Team. -21. :doc:`MBart ` (from Facebook) released with the paper `Multilingual Denoising Pre-training for - Neural Machine Translation `__ by Yinhan Liu, Jiatao Gu, Naman Goyal, Xian Li, - Sergey Edunov, Marjan Ghazvininejad, Mike Lewis, Luke Zettlemoyer. -22. :doc:`Pegasus ` (from Google) released with the paper `PEGASUS: Pre-training with Extracted - Gap-sentences for Abstractive Summarization `__> by Jingqing Zhang, Yao Zhao, - Mohammad Saleh and Peter J. Liu. -23. :doc:`Reformer ` (from Google Research) released with the paper `Reformer: The Efficient - Transformer `__ by Nikita Kitaev, Łukasz Kaiser, Anselm Levskaya. -24. :doc:`RoBERTa ` (from Facebook), released together with the paper a `Robustly Optimized BERT - Pretraining Approach `__ by Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar - Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, Veselin Stoyanov. ultilingual BERT into `DistilmBERT +8. `DialoGPT `__ (from Microsoft Research) released with + the paper `DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation + `__ by Yizhe Zhang, Siqi Sun, Michel Galley, Yen-Chun Chen, Chris Brockett, Xiang + Gao, Jianfeng Gao, Jingjing Liu, Bill Dolan. +9. `DistilBERT `__ (from HuggingFace), released together + with the paper `DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter + `__ by Victor Sanh, Lysandre Debut and Thomas Wolf. The same method has been + applied to compress GPT2 into `DistilGPT2 + `__, RoBERTa into `DistilRoBERTa + `__, Multilingual BERT into + `DistilmBERT `__ and a German version + of DistilBERT. +10. `DPR `__ (from Facebook) released with the paper `Dense Passage Retrieval + for Open-Domain Question Answering `__ by Vladimir Karpukhin, Barlas Oğuz, Sewon + Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. +11. `ELECTRA `__ (from Google Research/Stanford University) + released with the paper `ELECTRA: Pre-training text encoders as discriminators rather than generators + `__ by Kevin Clark, Minh-Thang Luong, Quoc V. Le, Christopher D. Manning. +12. `FlauBERT `__ (from CNRS) released with the paper + `FlauBERT: Unsupervised Language Model Pre-training for French `__ by Hang Le, + Loïc Vial, Jibril Frej, Vincent Segonne, Maximin Coavoux, Benjamin Lecouteux, Alexandre Allauzen, Benoît Crabbé, + Laurent Besacier, Didier Schwab. +13. `Funnel Transformer `__ (from CMU/Google Brain) released with the + paper `Funnel-Transformer: Filtering out Sequential Redundancy for Efficient Language Processing + `__ by Zihang Dai, Guokun Lai, Yiming Yang, Quoc V. Le. +14. `GPT `__ (from OpenAI) released with the paper `Improving + Language Understanding by Generative Pre-Training `__ by Alec + Radford, Karthik Narasimhan, Tim Salimans and Ilya Sutskever. +15. `GPT-2 `__ (from OpenAI) released with the paper `Language + Models are Unsupervised Multitask Learners `__ by Alec Radford*, + Jeffrey Wu*, Rewon Child, David Luan, Dario Amodei** and Ilya Sutskever**. +16. `LayoutLM `__ (from Microsoft Research Asia) released with + the paper `LayoutLM: Pre-training of Text and Layout for Document Image Understanding + `__ by Yiheng Xu, Minghao Li, Lei Cui, Shaohan Huang, Furu Wei, Ming Zhou. +17. `Longformer `__ (from AllenAI) released with the + paper `Longformer: The Long-Document Transformer `__ by Iz Beltagy, Matthew E. + Peters, Arman Cohan. +18. `LXMERT `__ (from UNC Chapel Hill) released with the paper `LXMERT: Learning + Cross-Modality Encoder Representations from Transformers for Open-Domain Question Answering + `__ by Hao Tan and Mohit Bansal. +19. `MarianMT `__ Machine translation models trained using + `OPUS `__ data by Jörg Tiedemann. The `Marian Framework `__ is + being developed by the Microsoft Translator Team. +20. `MBart `__ (from Facebook) released with the paper + `Multilingual Denoising Pre-training for Neural Machine Translation `__ by Yinhan + Liu, Jiatao Gu, Naman Goyal, Xian Li, Sergey Edunov, Marjan Ghazvininejad, Mike Lewis, Luke Zettlemoyer. +21. `MMBT `__ (from Facebook), released together with the paper a + `Supervised Multimodal Bitransformers for Classifying Images and Text `__ by + Douwe Kiela, Suvrat Bhooshan, Hamed Firooz, Davide Testuggine. +22. `Pegasus `__ (from Google) released with the paper `PEGASUS: + Pre-training with Extracted Gap-sentences for Abstractive Summarization `__> by + Jingqing Zhang, Yao Zhao, Mohammad Saleh and Peter J. Liu. +23. `Reformer `__ (from Google Research) released with the + paper `Reformer: The Efficient Transformer `__ by Nikita Kitaev, Łukasz Kaiser, + Anselm Levskaya. +24. `RoBERTa `__ (from Facebook), released together with + the paper a `Robustly Optimized BERT Pretraining Approach `__ by Yinhan Liu, Myle + Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, Veselin Stoyanov. + ultilingual BERT into `DistilmBERT `__ and a German version of DistilBERT. -25. `SqueezeBert `__ released with the paper +25. `SqueezeBert `__ released with the paper `SqueezeBERT: What can computer vision teach NLP about efficient neural networks? `__ by Forrest N. Iandola, Albert E. Shaw, Ravi Krishna, and Kurt W. Keutzer. -26. :doc:`T5 ` (from Google AI) released with the paper `Exploring the Limits of Transfer Learning with a - Unified Text-to-Text Transformer `__ by Colin Raffel and Noam Shazeer and Adam - Roberts and Katherine Lee and Sharan Narang and Michael Matena and Yanqi Zhou and Wei Li and Peter J. Liu. -27. :doc:`Transformer-XL ` (from Google/CMU) released with the paper `Transformer-XL: - Attentive Language Models Beyond a Fixed-Length Context `__ by Zihang Dai*, - Zhilin Yang*, Yiming Yang, Jaime Carbonell, Quoc V. Le, Ruslan Salakhutdinov. -28. :doc:`XLM ` (from Facebook) released together with the paper `Cross-lingual Language Model - Pretraining `__ by Guillaume Lample and Alexis Conneau. -29. :doc:`XLM-RoBERTa ` (from Facebook AI), released together with the paper `Unsupervised - Cross-lingual Representation Learning at Scale `__ by Alexis Conneau*, Kartikay - Khandelwal*, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke - Zettlemoyer and Veselin Stoyanov. -30. :doc:`XLNet ` (from Google/CMU) released with the paper `​XLNet: Generalized Autoregressive - Pretraining for Language Understanding `__ by Zhilin Yang*, Zihang Dai*, Yiming - Yang, Jaime Carbonell, Ruslan Salakhutdinov, Quoc V. Le. +26. `T5 `__ (from Google AI) released with the paper `Exploring + the Limits of Transfer Learning with a Unified Text-to-Text Transformer `__ by + Colin Raffel and Noam Shazeer and Adam Roberts and Katherine Lee and Sharan Narang and Michael Matena and Yanqi + Zhou and Wei Li and Peter J. Liu. +27. `Transformer-XL `__ (from Google/CMU) released + with the paper `Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context + `__ by Zihang Dai*, Zhilin Yang*, Yiming Yang, Jaime Carbonell, Quoc V. Le, + Ruslan Salakhutdinov. +28. `XLM `__ (from Facebook) released together with the paper + `Cross-lingual Language Model Pretraining `__ by Guillaume Lample and Alexis + Conneau. +29. `XLM-RoBERTa `__ (from Facebook AI), released + together with the paper `Unsupervised Cross-lingual Representation Learning at Scale + `__ by Alexis Conneau*, Kartikay Khandelwal*, Naman Goyal, Vishrav Chaudhary, + Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer and Veselin Stoyanov. +30. `XLNet `__ (from Google/CMU) released with the paper + `​XLNet: Generalized Autoregressive Pretraining for Language Understanding `__ by + Zhilin Yang*, Zihang Dai*, Yiming Yang, Jaime Carbonell, Ruslan Salakhutdinov, Quoc V. Le. 31. `Other community models `__, contributed by the `community `__. diff --git a/setup.py b/setup.py index 0363c23e24..38cb92861a 100644 --- a/setup.py +++ b/setup.py @@ -3,9 +3,7 @@ Simple check list from AllenNLP repo: https://github.com/allenai/allennlp/blob/m To create the package for pypi. -1. Change the version in __init__.py, setup.py as well as docs/source/conf.py. Remove the master from the links in - the new models of the README: - (https://huggingface.co/transformers/master/model_doc/ -> https://huggingface.co/transformers/model_doc/) +1. Change the version in __init__.py, setup.py as well as docs/source/conf.py. 2. Unpin specific versions from setup.py that use a git install. diff --git a/utils/check_copies.py b/utils/check_copies.py index 264c914e0b..1a7fcfe04b 100644 --- a/utils/check_copies.py +++ b/utils/check_copies.py @@ -24,7 +24,6 @@ import tempfile # python utils/check_copies.py TRANSFORMERS_PATH = "src/transformers" PATH_TO_DOCS = "docs/source" -REPO_PATH = "." def find_code_in_transformers(object_name): @@ -176,7 +175,7 @@ def get_model_list(): # If the introduction or the conclusion of the list change, the prompts may need to be updated. _start_prompt = "🤗 Transformers currently provides the following architectures" _end_prompt = "1. Want to contribute a new model?" - with open(os.path.join(REPO_PATH, "README.md"), "r", encoding="utf-8") as f: + with open(os.path.join("README.md"), "r", encoding="utf-8") as f: lines = f.readlines() # Find the start of the list. start_index = 0 @@ -220,17 +219,7 @@ def split_long_line_with_indent(line, max_per_line, indent): def convert_to_rst(model_list, max_per_line=None): """ Convert `model_list` to rst format. """ # Convert **[description](link)** to `description `__ - def _rep_link(match): - title, link = match.groups() - # Keep hard links for the models not released yet - if "master" in link or not link.startswith("https://huggingface.co/transformers"): - return f"`{title} <{link}>`__" - # Convert links to relative links otherwise - else: - link = link[len("https://huggingface.co/transformers/") : -len(".html")] - return f":doc:`{title} <{link}>`" - - model_list = re.sub(r"\*\*\[([^\]]*)\]\(([^\)]*)\)\*\*", _rep_link, model_list) + model_list = re.sub(r"\*\*\[([^\]]*)\]\(([^\)]*)\)\*\*", r"`\1 <\2>`__", model_list) # Convert [description](link) to `description `__ model_list = re.sub(r"\[([^\]]*)\]\(([^\)]*)\)", r"`\1 <\2>`__", model_list)