Skip to content

Instantly share code, notes, and snippets.

@fgassert

fgassert/gcp_dataupload

Last active August 29, 2015 13:56
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save fgassert/8928606 to your computer and use it in GitHub Desktop.
Save fgassert/8928606 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
gcp_dataupload
#!/bin/bash -x
S3DIR=s3://wri-dataupload/isciences/CMIP5/yr/
FILENAME=CMIP5_annual_20140210
PUTDIR=s3://wri-public-data/Aqueduct/
curl -O https://gist.github.com/fgassert/8928302/raw/s3cfg_from_env.sh
chmod +x s3cfg_from_env.sh
./s3cfg_from_env.sh
sudo apt-get install git -y
git clone https://github.com/s3tools/s3cmd.git
cd s3cmd
sudo python setup.py install
cd ..
mkdir $FILENAME
s3cmd -c .s3cfg get -r $S3DIR ./$FILENAME
tar -zcf $FILENAME.tgz $FILENAME
s3cmd -c .s3cfg put $FILENAME.tgz $PUTDIR
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment