SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity

Fix mixed precision in TF models (#9163)

Browse Source 
* Fix Gelu precision

* Fix gelu_fast

* Naming

* Fix usage and apply style

* add TF gelu approximate version

* add TF gelu approximate version

* add TF gelu approximate version

* Apply style

* Fix albert

* Remove the usage of the Activation layer
This commit is contained in:
Julien Plu
2021-01-21 13:00:11 +01:00
committed by GitHub
parent 248fa1ae72
commit 3f290e6c84
8 changed files with 28 additions and 15 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
src/transformers/models/bert/modeling_tf_bert.py
View File

@@ -428,7 +428,7 @@ class TFBertIntermediate(tf.keras.layers.Layer):
def call(self, hidden_states):
hidden_states = self.dense(inputs=hidden_states)
hidden_states = self.intermediate_act_fn(inputs=hidden_states)
hidden_states = self.intermediate_act_fn(hidden_states)
return hidden_states