SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity

Replace "Masked" with "Causal" in TF CLM example (#14014)

Browse Source
This commit is contained in:
Christopher Akiki
2021-10-21 17:19:30 +02:00
committed by GitHub
parent 3187228206
commit f9c16b02e3
1 changed files with 3 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
examples/tensorflow/language-modeling/run_clm.py
View File

@@ -14,13 +14,13 @@
# See the License for the specific language governing permissions and # See the License for the specific language governing permissions and
# limitations under the License. # limitations under the License.
""" """
Fine-tuning the library models for masked language modeling (BERT, ALBERT, RoBERTa...) Fine-tuning the library models for causal language modeling (GPT-2, GPT-Neo...)
on a text file or a dataset without using HuggingFace Trainer. on a text file or a dataset without using HuggingFace Trainer.
Here is the full list of checkpoints on the hub that can be fine-tuned by this script: Here is the full list of checkpoints on the hub that can be fine-tuned by this script:
https://huggingface.co/models?filter=masked-lm https://huggingface.co/models?filter=causal-lm
""" """
# You can also adapt this script on your own mlm task. Pointers for this are left as comments. # You can also adapt this script on your own clm task. Pointers for this are left as comments.
# region Imports # region Imports
import logging import logging