Skip to content

Instantly share code, notes, and snippets.

@foemre
Created November 29, 2020 11:19
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save foemre/b0cb9af8bf5a7e6b4978ad4f35af05b2 to your computer and use it in GitHub Desktop.
Save foemre/b0cb9af8bf5a7e6b4978ad4f35af05b2 to your computer and use it in GitHub Desktop.
google colab env prepare py3.7
#!/bin/bash
sudo apt install -y build-essential zlib1g-dev libncurses5-dev libgdbm-dev libnss3-dev libssl-dev libreadline-dev libffi-dev wget python3.7 python3-pip python3.7-dev
git clone https://github.com/grammarly/gector.git
cd gector
wget https://www.cl.cam.ac.uk/research/nl/bea2019st/data/fce_v2.1.bea19.tar.gz
wget https://www.cl.cam.ac.uk/research/nl/bea2019st/data/corr_from_m2.py
tar -xzf fce_v2.1.bea19.tar.gz
python3.7 -m pip install -r requirements.txt
cat fce/m2/fce.train.gold.bea19.m2 | grep "^S .*" > train_orig.txt
cat fce/m2/fce.test.gold.bea19.m2 | grep "^S .*" > test_orig.txt
cat fce/m2/fce.dev.gold.bea19.m2 | grep "^S .*" > dev_orig.txt
python3.7 corr_from_m2.py -out train_corr.txt fce/m2/fce.train.gold.bea19.m2
python3.7 corr_from_m2.py -out test_corr.txt fce/m2/fce.test.gold.bea19.m2
python3.7 corr_from_m2.py -out dev_corr.txt fce/m2/fce.dev.gold.bea19.m2
python3.7 utils/preprocess_data.py -s train_orig.txt -t train_corr.txt -o train_out.txt
python3.7 utils/preprocess_data.py -s dev_orig.txt -t dev_corr.txt -o dev_out.txt
python3.7 utils/preprocess_data.py -s test_orig.txt -t test_corr.txt -o test_out.txt
# python3.7 train.py --train_set train_out.txt --dev_set dev_out.txt --model_dir model --batch_size 32 --lr 5e-6 --cold_lr 5e-4 --n_epoch 50 --transformer_model distilbert
# python3.7 train.py --train_set train_out.txt --dev_set dev_out.txt --model_dir model --batch_size 16 --lr 1e-4 --cold_lr 1e-2 --n_epoch 50 --transformer_model distilbert
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment