Apache Mesos Notes

Mesos Getting Started.md

##Mesos High level notes

It takes about 15 minutes to stand up a cluster. https://elastic.mesosphere.io/

Once the cluster is available you'll get an email with some links to your new pool of resources

Lots of details here

Hadoop

It should take about 15 minute to stand up a Hadoop Cluster on Mesos also http://mesosphere.io/learn/run-hadoop-on-mesos-using-installer/

SAD PANDA

I lost my connection to the terminal in the middle of the the hadoop initialization and now it won't run again, apparently it is not idempotent. Hadoop now shows up as a terminated instance in the mesos console.

Rails

http://mesosphere.io/learn/run-ruby-on-rails-on-mesos/

Although I'm not sure I'd target a platform like Mesos for long running apps I tried to run this tutorial. Rails requires Marathon which is a platform for long running processes running on Mesos. Once you install Mesos you'll be able to bring up it's dashboard at :8080.

While they have fast bootstrap process for the cluster itself there is still a lot of boilerplate copy pasta that they walk you through in the tutorials. For example, the command that preps the slaves for rails:

 servers=(slave1.example.com slave2.example.com slave3.example.com); for s in "${servers[@]}"; do ssh -l ubuntu $s 'sudo apt-get -y install ruby2.0 && sudo gem install bundler' ; done

Launching an ap from marathon was pretty nice But this was a trivial rails app with a single gem dependency and they hand held you through it.But the autoscaling feature is aweome, Click Scale, decide how many instances you want, profit.

While the toturial was fast an dthe end result pretty neat as you can see here Screen Cast

The tutorial doesn't leave you with a solution for proxying amongst your running instances but it does provide you with [script](https://github.com/mesosphere/marathon/blob/master/bin/haproxy_cfg] and leaves the configuration of haproxy for extra credit

ETL, Chronos and Hadoop(again)

Chronos is basically a pretty crontab with a nice interface. This tutorial went through booting up chronos to schedule ETL jobs across a hadoop cluster. Including getting the Hadoop client up and running it took about 20 minutes to complete. The process wasn't very telling because it was just another hand held tutorial but I've manageed a large scale cluster of ETL nodes before and this would have been really nice to have.

##Thoughts

• The user manaterials seem like a good start with a lot of helpful information but they aren't really automating much aside from starting the mesos cluster. Everythign else is made to look simple via tutorial or via a script that will require editing for your individual case.
• If you need High availabiliy or fast scaling and have a relatively simple application requirement or simple purpose cluster Mesos seems pretty slick.
• The main Mesos interface is really only a dashboard. You can't interact with your processes from the web interface but it does give you some useful information. I have not yet confirmed that the information is correct/valid but...
• There are some interesting atrifacts from my failed hadoop instance in the interface. The Dashboard says the hadoop process terminated but the processes say they are still running.. Odd.
• The interface does allow you to view the aggregate errors, this woudl be a cool feature on its own for CF, is there a web interface for Logs?
• The tutorials are nice and the auto fill cluster information feature would be really nice for CF documentation.