From 14c79c3e31901e0361827289c859707ae9be3422 Mon Sep 17 00:00:00 2001 From: Stas Bekman Date: Tue, 15 Dec 2020 06:10:41 -0800 Subject: [PATCH] native amp leak fix landed in 1.7.1 (#9115) update README with good news that the leak fix has been applied to pytorch-1.7.1. --- examples/seq2seq/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/examples/seq2seq/README.md b/examples/seq2seq/README.md index c2860ae1ab..cd4dcd6a0d 100644 --- a/examples/seq2seq/README.md +++ b/examples/seq2seq/README.md @@ -97,7 +97,7 @@ The `.source` files are the input, the `.target` files are the desired output. ### Potential issues -- native AMP (`--fp16` and no apex) may lead to a huge memory leak and require 10x gpu memory. This has been fixed in pytorch-nightly and the minimal official version to have this fix will be pytorch-1.8. Until then if you have to use mixed precision please use AMP only with pytorch-nightly or NVIDIA's apex. Reference: https://github.com/huggingface/transformers/issues/8403 +- native AMP (`--fp16` and no apex) may lead to a huge memory leak and require 10x gpu memory. This has been fixed in pytorch-nightly and the minimal official version to have this fix will be pytorch-1.7.1. Until then if you have to use mixed precision please use AMP only with pytorch-nightly or NVIDIA's apex. Reference: https://github.com/huggingface/transformers/issues/8403 ### Tips and Tricks