SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity

Added mask_time_prob and mask_time_length arguments to wav2vec2 pretraining script (#20985)

Browse Source 
Added mask_time_prob and mask_time_length arguments to wav2vec2 pretraining script and readme - new branch
This commit is contained in:
Magnus Pierrau
2023-01-05 17:24:55 +01:00
committed by GitHub
parent bc53fc6265
commit 1d21471c78
2 changed files with 45 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
examples/pytorch/speech-pretraining/README.md
View File

@@ -79,6 +79,8 @@ accelerate launch run_wav2vec2_pretraining_no_trainer.py \
--adam_beta2="0.98" \
--adam_epsilon="1e-06" \
--gradient_checkpointing \
--mask_time_prob="0.65" \
--mask_time_length="10"
```
The results of this run can be seen [here](https://wandb.ai/patrickvonplaten/wav2vec2-pretrained-demo/reports/Wav2Vec2-PreTraining-Demo-Run--VmlldzoxMDk3MjAw?accessToken=oa05s1y57lizo2ocxy3k01g6db1u4pt8m6ur2n8nl4cb0ug02ms2cw313kb8ruch).
@@ -110,6 +112,8 @@ accelerate launch run_wav2vec2_pretraining_no_trainer.py \
--adam_beta2="0.98" \
--adam_epsilon="1e-06" \
--gradient_checkpointing \
--mask_time_prob="0.65" \
--mask_time_length="10"
```
The experiment was run on 8 GPU V100 (16 GB RAM each) for 4 days.
@@ -146,6 +150,8 @@ accelerate launch run_wav2vec2_pretraining_no_trainer.py \
--adam_beta2=0.98 \
--adam_epsilon=1e-06 \
--gradient_checkpointing \
--mask_time_prob=0.65 \
--mask_time_length=10
```
The experiment was run on 8 GPU V100 (16 GB RAM each) for 7 days.