SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity

add blog to docs (#10997)

Browse Source
This commit is contained in:
Patrick von Platen
2021-03-31 18:36:00 +03:00
committed by GitHub
parent cd56f3fe7e
commit 01068abdb9
1 changed files with 2 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
docs/source/model_doc/bigbird.rst
View File

@@ -41,6 +41,8 @@ propose novel applications to genomics data.*
Tips: Tips:
- For an in-detail explanation on how BigBird's attention works, see `this blog post
<https://huggingface.co/blog/big-bird>`__.
- BigBird comes with 2 implementations: **original_full** & **block_sparse**. For the sequence length < 1024, using - BigBird comes with 2 implementations: **original_full** & **block_sparse**. For the sequence length < 1024, using
**original_full** is advised as there is no benefit in using **block_sparse** attention. **original_full** is advised as there is no benefit in using **block_sparse** attention.
- The code currently uses window size of 3 blocks and 2 global blocks. - The code currently uses window size of 3 blocks and 2 global blocks.