SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity

Depreciate pythonic Mish and support PyTorch 1.9 version of Mish (#12240)

Browse Source 
* Moved Mish to Torch 1.9 version

* Run black formatting
This commit is contained in:
Xa9aX ツ
2021-06-18 18:43:45 +05:30
committed by GitHub
parent 47a9768334
commit f3558bbcfd
3 changed files with 11 additions and 9 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
templates/adding_a_new_model/cookiecutter-template-{{cookiecutter.modelname}}/modeling_{{cookiecutter.lowercase_modelname}}.py
View File

@@ -138,10 +138,6 @@ def load_tf_weights_in_{{cookiecutter.lowercase_modelname}}(model, config, tf_ch
return model
def mish(x):
return x * torch.tanh(nn.functional.softplus(x))
# Copied from transformers.models.bert.modeling_bert.BertEmbeddings with Bert->{{cookiecutter.camelcase_modelname}}
class {{cookiecutter.camelcase_modelname}}Embeddings(nn.Module):
"""Construct the embeddings from word, position and token_type embeddings."""