From 5442a11f5f9b4fc0ff58a8c6fbfb6ee0c8812425 Mon Sep 17 00:00:00 2001 From: yylun <273971776@qq.com> Date: Wed, 3 Feb 2021 22:30:37 +0800 Subject: [PATCH] fix steps_in_epoch variable in trainer when using max_steps (#9969) * fix steps_in_epoch variable when using max_steps * redundant sentence * Revert "redundant sentence" This reverts commit ad5c0e9b6e66d65732dee2239cdc9c76dfa0dc5a. * remove redundant sentence Co-authored-by: wujindou --- README.md | 1 - docs/source/index.rst | 4 +--- src/transformers/trainer.py | 6 +++++- 3 files changed, 6 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index ad2229364e..fdb9ad317a 100644 --- a/README.md +++ b/README.md @@ -223,7 +223,6 @@ Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 1. **[ProphetNet](https://huggingface.co/transformers/model_doc/prophetnet.html)** (from Microsoft Research) released with the paper [ProphetNet: Predicting Future N-gram for Sequence-to-Sequence Pre-training](https://arxiv.org/abs/2001.04063) by Yu Yan, Weizhen Qi, Yeyun Gong, Dayiheng Liu, Nan Duan, Jiusheng Chen, Ruofei Zhang and Ming Zhou. 1. **[Reformer](https://huggingface.co/transformers/model_doc/reformer.html)** (from Google Research) released with the paper [Reformer: The Efficient Transformer](https://arxiv.org/abs/2001.04451) by Nikita Kitaev, Łukasz Kaiser, Anselm Levskaya. 1. **[RoBERTa](https://huggingface.co/transformers/model_doc/roberta.html)** (from Facebook), released together with the paper a [Robustly Optimized BERT Pretraining Approach](https://arxiv.org/abs/1907.11692) by Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, Veselin Stoyanov. -ultilingual BERT into [DistilmBERT](https://github.com/huggingface/transformers/tree/master/examples/distillation) and a German version of DistilBERT. 1. **[SqueezeBert](https://huggingface.co/transformers/model_doc/squeezebert.html)** released with the paper [SqueezeBERT: What can computer vision teach NLP about efficient neural networks?](https://arxiv.org/abs/2006.11316) by Forrest N. Iandola, Albert E. Shaw, Ravi Krishna, and Kurt W. Keutzer. 1. **[T5](https://huggingface.co/transformers/model_doc/t5.html)** (from Google AI) released with the paper [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://arxiv.org/abs/1910.10683) by Colin Raffel and Noam Shazeer and Adam Roberts and Katherine Lee and Sharan Narang and Michael Matena and Yanqi Zhou and Wei Li and Peter J. Liu. 1. **[TAPAS](https://huggingface.co/transformers/model_doc/tapas.html)** (from Google AI) released with the paper [TAPAS: Weakly Supervised Table Parsing via Pre-training](https://arxiv.org/abs/2004.02349) by Jonathan Herzig, Paweł Krzysztof Nowak, Thomas Müller, Francesco Piccinno and Julian Martin Eisenschlos. diff --git a/docs/source/index.rst b/docs/source/index.rst index b1c4af7168..b28ec05b6b 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -177,9 +177,7 @@ and conversion utilities for the following models: Transformer `__ by Nikita Kitaev, Łukasz Kaiser, Anselm Levskaya. 32. :doc:`RoBERTa ` (from Facebook), released together with the paper a `Robustly Optimized BERT Pretraining Approach `__ by Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar - Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, Veselin Stoyanov. ultilingual BERT into `DistilmBERT - `__ and a German version of - DistilBERT. + Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, Veselin Stoyanov. 33. :doc:`SqueezeBert ` released with the paper `SqueezeBERT: What can computer vision teach NLP about efficient neural networks? `__ by Forrest N. Iandola, Albert E. Shaw, Ravi Krishna, and Kurt W. Keutzer. diff --git a/src/transformers/trainer.py b/src/transformers/trainer.py index f73d7d0007..a08938caa7 100755 --- a/src/transformers/trainer.py +++ b/src/transformers/trainer.py @@ -910,7 +910,11 @@ class Trainer: if self.args.past_index >= 0: self._past = None - steps_in_epoch = len(epoch_iterator) if train_dataset_is_sized else self.args.max_steps + steps_in_epoch = ( + len(epoch_iterator) + if train_dataset_is_sized + else self.args.max_steps * self.args.gradient_accumulation_steps + ) self.control = self.callback_handler.on_epoch_begin(self.args, self.state, self.control) for step, inputs in enumerate(epoch_iterator):