Skip to content

Instantly share code, notes, and snippets.

@byllc
Last active August 29, 2015 13:57
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save byllc/9392792 to your computer and use it in GitHub Desktop.
Save byllc/9392792 to your computer and use it in GitHub Desktop.
Mesos Technical Overview

###Apache Mesos A cluster manager for high availabilty distributed applications. The main components of the Mesos arechitecture are master daemon that manages slave daemons running on each cluster node, and mesos applications (frameworks) that run on the slaves. The authors define Mesos as "A thin layer for resource sharing"

The master distributes resources (cpu, ram etc.) across applications by making them resource offers. The master uses an organizational policy to decide how many resources to allow a framework.

A framework running on top of Mesos consists a scheduler that registers with the master to be offered resources, and an executor process that is launched on slave nodes to run the framework’s tasks. When a frameworks accepts resources offered by the master it tells mesos what it would like to run with those resoureses and mesos launces the tasks on the slaves.

More about Executors

####Packaging Thus far I have not found details on a concrete packaging construct. While the framework provides for scheduling and execution of a process it does not appear to dictate how you package your process for distribution. You provide your schedule with details about where to find the package source (as tarball) and a commant to run against the decompressed source. As in the rails example you may also need to prep your slaves separately with dependencies (Like Ruby and bundler in the Rails example).

####Slave Recovery Mesos supports slave recovery via checkpointing. You must enable this feature by setting checkpoint=true on FrameowrkInfo. Checkpointing must be enabled on the slave and the framework must have checkpointing enabled. Because this feature create disk io overhead you can opt out.

More details here

####Finding Resources for an Executor Mesos uses a construct called "resource offers". Mesos offeres a framework a set of resources and the framework decides which resources to use and what to run on them. These offers can be controled via organizational policies that can be created and adjusted for each organization's needs.

####High Availability In order to provide HA a zookeeper cluster must be running. Zookeeper is built in to Mesos, the cluster is a set of available master nodes.

##Useful Links [Tutorials[(http://mesosphere.io/learn/) Community

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment