Removes images to put them in a dataset (#14781)

* First try

* Update instructions
This commit is contained in:
Lysandre Debut
2021-12-16 04:42:02 -05:00
committed by GitHub
parent 459677aebe
commit 8010fda9bf
38 changed files with 46 additions and 36 deletions

View File

@@ -248,7 +248,7 @@ Here are the commonly used floating point data types choice of which impacts bot
Here is a diagram that shows how these data types correlate to each other.
![data types](/imgs/tf32-bf16-fp16-fp32.png)
![data types](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/tf32-bf16-fp16-fp32.png)
(source: [NVIDIA Blog](https://developer.nvidia.com/blog/accelerating-ai-training-with-tf32-tensor-cores/))
@@ -524,7 +524,7 @@ Since it has been discovered that more parameters lead to better performance, th
In this approach every other FFN layer is replaced with a MoE Layer which consists of many experts, with a gated function that trains each expert in a balanced way depending on the input token's position in a sequence.
![MoE Transformer 2x block](/imgs/perf-moe-transformer.png)
![MoE Transformer 2x block](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/perf-moe-transformer.png)
(source: [GLAM](https://ai.googleblog.com/2021/12/more-efficient-in-context-learning-with.html))