===
- I was running a AWS instence to host a Minecraft Map ( http://map.ob3y.com/mcmap ) with friends and it was costing me a bunch of $$$
- Rendering on a cluster was slow, since disk was a bit slow, and MC Overviewer would only use 1 single note (no parrallel feature)
- rsyncing from the "map processing" nodes to the "web node" was slooooo
- I ran out of disk space on the web node ... booo... and this equals small maps...
===
===
We've been working on some cloud POC stuff at work, so I decided to see if I could use some of that to make my life better. took a fork of our generic vagrant box https://github.com/huit/vagrant-generic and created a Minecraft Overviewer specific config.
===
===
s3 and how the heck to use it.. curl, wget.. boto...
First I looked at the aws cli tools to push and copy but they seemed like a bit of a pain for anything but 1/2 files Next I looked at boto, and figured I'd just write my own. Then I ran across s2cmd tools... whoa I also came across a cool s3file type (link needed) for puppet.. whoa mind blowing
===
make sure you use an instance thats large enough
run as root - /media/ephemeral isn't accessable to ec2-user s3cmd get s3://bungalow/Bungalow.zip
Data transfers have been crazy fast
make sure you have a workspace where your ec2-user has permisions to work. I had puppet create /media/ephemeral0/workspace
clone the configs git repo git clone https://github.com/tfhartmann/mc-overviewer-configs.git configs
Run the overviewer render app
overviewer.py --verbose --config=/media/ephemeral0/workspace/configs/Bungalow.cfg
assetmanager.py:66 12018 2013-08-09 16:19:22 tileset.py:798 12018 2013-08-09 16:28:51 DEBUG Finished chunk scan for Bungalowday. 1488686 chunks scanned in 535 seconds
Which is actually pretty close in time to what I was getting before
2013-07-30 13:20:40 DEBUG Finished chunk scan for Bungalowday. 1488396 chunks scanned in 625 seconds
screen -d -m overviewer.py --verbose --config=/media/ephemeral0/workspace/configs/Bungalow.cfg
===
Internal to our net
Bungalow.zip 100% 3504MB 33.4MB/s 01:45
s3://bungalow/Bungalow.zip -> ./Bungalow.zip [1 of 1] 3673890313 of 3673890313 100% in 141s 24.82 MB/s done
so 1min 45 seconds internal and 2min 21 seconds in AWS
Instance Compare
ec2_instance_type => m2.4xlarge
physicalprocessorcount => 1
processor7 => Intel(R) Xeon(R) CPU X5550 @ 2.67GHz
processorcount => 8This one looks like it'll take about 17/18 hours to complete with about an hour ramp up. So that 17/18 hours should be at about 100% CPU Usage with an hour of warmup for the app
===
====
https://github.com/tfhartmann/vagrant-generic/tree/minecraft-overviewer
https://github.com/huit/vagrant-generic
====