Skip to content

Instantly share code, notes, and snippets.

@myleott

myleott/fairseq_benchmark_masked_lm.c10d.200mb.sh

Created August 2, 2019 16:42
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save myleott/804c07a92a49e9838ef50de008b1acd4 to your computer and use it in GitHub Desktop.
Save myleott/804c07a92a49e9838ef50de008b1acd4 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
fairseq_benchmark_masked_lm.c10d.200mb.sh
#!/bin/bash
source activate fairseq
fairseq-train --no-save --disable-validation --task dummy_masked_lm --masked-lm-only --arch bert_large --num-segment 0 --optimizer adam --lr 1e-4 --max-sentences 8 --update-freq 1 --fp16 --ddp-backend $DDP_BACKEND --bucket-cap-mb $BUCKET_CAP_MB --tokens-per-sample 512 --dataset-size $DATASET_SIZE --criterion masked_lm_loss --log-format json --log-interval 10 --max-epoch 1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment