Uploading Big Data to AWS

data-upload.Rmd

I had to do this today and thought of our discussion. I figured I'd document the commands and send them over - maybe it'll help.

http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-using-volumes.html

Commands:

$ sudo su

# lsblk
NAME    MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
xvda    202:0    0   8G  0 disk
└─xvda1 202:1    0   8G  0 part /
xvdb    202:16   0   5G  0 disk

# file -s /dev/xvdb
/dev/xvdb: data

# mkfs -t ext4 /dev/xvdb
...

# mkdir /mnt/web

# mount /dev/xvdb /mnt/web

# df -h

...
/dev/xvdb       4.8G   10M  4.6G   1% /mnt/web

# echo '/dev/xvdb /mnt/web ext4 defaults,nofail 0 2' >> /etc/fstab

# reboot

Wait about a minute or so and try logging in... then type:

$ df -h

...
/dev/xvdb       4.8G   10M  4.6G   1% /mnt/web

if you see that last line ... it mounted and all is well.

Obviously you should change out 'web' for whatever  you'd like your directory to be. I usually set up a symbolic link to the directory as part of my file structure. For instance:

$ cd /var; ln -s /mnt/web

creates a directory /var/web on my file structure. If needed, I can change that directory to a new mount point easily - your mileage may vary.

@yxes Thanks for this.