SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity

[Benchmark] Add benchmarks for TF Training (#5594)

Browse Source 
* tf_train

* adapt timing for tpu

* fix timing

* fix timing

* fix timing

* fix timing

* update notebook

* add tests
This commit is contained in:
Patrick von Platen
2020-07-08 12:11:09 +02:00
committed by GitHub
parent cfbb982974
commit f82a2a5e8e
4 changed files with 103 additions and 15 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
notebooks/05-benchmark.ipynb
View File

@@ -312,8 +312,8 @@
":-- | :--- | :--- | :--- | :--- | :--- | :--- | :--- |\n",
"**Speed - Inference** | ✔ | ✔ | ✔ | ✔ | ✔ | ✘ | ✔ |\n",
"**Memory - Inference** | ✔ | ✔ | ✔ | ✔ | ✔ | ✘ | ✘ |\n",
"**Speed - Train** | ✘ | ✘ | ✘ | ✘ | ✘ | ✘ | ✘ |\n",
"**Memory - Train** | ✘ | ✘ | ✘ | ✘ | ✘ | ✘ | ✘ |\n",
"**Speed - Train** | ✔ | ✘ | ✔ | ✘ | ✘ | ✘ | ✔ |\n",
"**Memory - Train** | ✔ | ✘ | ✔ | ✘ | ✘ | ✘ | ✘ |\n",
"\n",
"* *eager execution* means that the function is run in the eager execution environment of TensorFlow 2, see [here](https://www.tensorflow.org/guide/eager).\n",
"\n",
@@ -321,7 +321,7 @@
"\n",
"* *FP16* stands for TensorFlow's mixed-precision package and is analogous to PyTorch's FP16 feature, see [here](https://www.tensorflow.org/guide/mixed_precision).\n",
"\n",
"***Note***: In ~1,2 weeks it will also be possible to benchmark training in TensorFlow.\n",
"***Note***: Benchmark training in TensorFlow is not included in v3.0.2, but available in master.\n",
"\n",
"\n",
"This notebook will show the user how to use `PyTorchBenchmark` and `TensorFlowBenchmark` for two different scenarios:\n",