dharmit/centos-mesos-marathon.md Secret

## centos-mesos-marathon.md

      
    Raw
  

              centos-mesos-marathon.md
            
          
    Contents:

Introduction
Setting up Mesos and Marathon on CentOS 7
Setup Atomicapp
Example: HelloApache
Setup Mesos-DNS
Example: Flask + Redis based page visit counter app

###Introduction
In this guide, we are going to setup Mesos and Marathon on a fresh CentOS 7 system. Then, we are going to check examples using atomicapp and the Marathon provider built into it. These examples help us deploy containerized application on Mesos using Marathon. Note that one can use Marathon to deploy non-containerized workload as well. We will also explore how to use mesos-dns when developing multi-container apps that talk to each other over the network.
In case you already have an environment setup, go to the first example right away.
###Setting up Mesos and Marathon on CentOS 7
I am going to use vagrant to demonstrate the setup. In case you don't know about vagrant, you can learn more about it from its homepage. You can install it by doing sudo dnf install vagrant on your Fedora 22/23 system.
Now, create a directory where you'll store the Vagrantfile. Below are the contents of our simple Vagrantfile:
# -*- mode: ruby -*-
# vi: set ft=ruby :
    
Vagrant.configure("2") do |config|
    config.vm.box = "centos/7"
    
    config.vm.provider "virtualbox" do |v| 
        v.memory = 2048
        v.cpus = 2 
    end
   
   config.vm.network "private_network", ip: "10.2.2.2"
   
end

Simply put, Vagrantfile is to vagrant what Dockerfile is to Docker. However, this is not 100% accurate statement and is just an analogy to help the reader.
Now, we'll boot our virtual machine (vm/box) for the first time. Depending on whether centos7 vagrant box is already downloaded on your system or not, below command will take anything from a few seconds to about half an hour to setup a virtual machine. Make sure to execute the command from the same directory that contains the Vagrantfile mentioned earlier:
$ vagrant up --provider virtualbox

I am using the virtualbox provider of Vagrant for this example. You can choose the one that you like. CentOS 7 vagrant boxes are available here.
Once the vm is setup, you can access its console by doing:
$ vagrant ssh

From inside the vm, first thing we do is the obligatory sudo yum -y update. This is not mandatory for our Mesos-Marathon setup to work. But it's the recommended thing to do after a new system is setup. And I've not tested things without the yum update.
Since packages related to Mesos and Marathon are available from the yum repo created by folks at Mesosphere, we need to enable the repo:
$ sudo rpm -Uvh http://repos.mesosphere.com/el/7/noarch/RPMS/mesosphere-el-repo-7-1.noarch.rpm

With Mesosphere repo enabled, we are good to install the necessary packages:
$ sudo yum -y install mesos marathon mesosphere-zookeeper

Above command will take some time depending on your Internet connection. Once installed, start and enable the services:
$ sudo systemctl start mesos-master
$ sudo systemctl start mesos-slave
$ sudo systemctl start zookeeper
$ sudo systemctl start marathon

$ sudo systemctl enable mesos-master
$ sudo systemctl enable mesos-slave
$ sudo systemctl enable zookeeper
$ sudo systemctl enable marathon

In the web browser on your host system, check if you're able to access Mesos master on http://10.2.2.2:5050/ and Marathon on http://10.2.2.2:8080. Note that if you specified a different IP than 10.2.2.2 in your Vagrantfile, you need to use that IP to check if Mesos master and Marathon are accessible from host system.
Mesos and Marathon should now be setup in your CentOS 7 system. Next, we will look at the examples.
Since the examples we will be looking at are about containerized applications, it is required to have Docker installed on the box. If you don't have it installed already, do:
$ sudo yum -y install docker

Start and enable the docker daemon:
$ sudo systemctl start docker
$ sudo systemctl enable docker

Instead of adding the concerned user to docker group, I prefer to put an alias in my ~/.zshrc like this:
alias docker="sudo docker"

If you use bash, put it in ~/.bashrc. Instead of exiting the terminal, just source the relevant file:
$ source ~/.zshrc

OR
$ source ~/.bashrc

More about it can be found on this blog on projectatomic.
Since we're going to deploy containers from Docker images, it is required to let Mesos slave know that we are going to use Docker as a containerizer. Make sure you don't miss this. I wasted almost an hour and half on debugging this issue when containers failed to start!
$ sudo sh -c "echo docker,mesos > /etc/mesos-slave/containerizers"
$ sudo systemctl restart mesos-slave

###Setup Atomicapp
To work on the examples, we need to ensure that atomicapp command is available on the vm. To install this, refer the GitHub page of atomicapp. Essentially, it involves executing below steps:
$ sudo yum -y install git epel-release 
$ sudo yum -y install python-pip  # not clubbed with above command as epel-release is a separate repo
$ git clone https://github.com/projectatomic/atomicapp
$ cd atomicapp
$ sudo pip install --upgrade pip  # optional
$ sudo pip install .

###Example: HelloApache
The first example we will be looking at is a simple apache page. You can find more about it on GitHub.
To run this example simply do:
$ sudo atomicapp run --provider=marathon tomaskral/helloapache

If docker.io/projectatomic/helloapache image is not present on your system, above command will trigger a docker pull for the same. Once the application is deployed, you can check the Marathon UI. Click on the application name in Marathon UI. It'll help you figure out the port on which the container is listening. In my case it was 31245. So I entered http://10.2.2.2:31245 in the web browser to see the apache welcome page.
That's all; we deployed a very basic example using atomicapp using Marathon provider.
###Setup Mesos-DNS
Mesos-DNS is a service discovery tool. Service Discovery is nicely summarized in this blog post. It doesn't cover Mesos-DNS. Mesos-DNS is specific to the Mesos environment.
To setup Mesos-DNS, download the binary from its releases page. Copy it to a location in the $PATH variable or modify the $PATH variable. Whatever is convenient to you. I copied to /usr/bin/:
$ sudo yum -y install wget bind-utils    # we'll need bind-utils for dig
$ wget https://github.com/mesosphere/mesos-dns/releases/download/v0.5.1/mesos-dns-v0.5.1-linux-amd64
$ sudo mv mesos-dns-v0.5.1-linux-amd64 /usr/bin/mesos-dns
$ sudo chmod +x /usr/bin/mesos-dns

Create a directory under /etc to store configuartion for Mesos-DNS. And create a file config.json under it with contents like below:
$ sudo mkdir /etc/mesos-dns
$ cat /etc/mesos-dns/config.json
{
    "zk": "zk://10.0.2.15:2181/mesos",
    "refreshSeconds": 60,
    "ttl": 60,
    "domain": "mesos",
    "port": 53,
    "resolvers": ["8.8.8.8"],
    "timeout": 5,
    "listener": "10.2.2.2",
    "httpon": true,
    "httpport": 8123
}

Replace 10.0.2.15 in above output to the IP of your vagrant box. This IP is of the system on which ZooKeeper is running.
Next, start Mesos-DNS from the commnad line or using Marathon. Mesos-DNS documentation suggests to use Marathon as it'll ensure that the Mesos-DNS service is restarted in event of any failure. I prefer checking from the command line before deploying it using Marathon.
$ sudo mesos-dns -config=/etc/mesos-dns/config.json

If Mesos-DNS was deployed successfully, you should be able to make DNS lookups using it as your server. Make sure to edit your /etc/resolv.conf to have the IP of system running Mesos-DNS as the first entry for nameserver:
$ cat /etc/resolv.conf
nameserver 10.2.2.2
nameserver 8.8.8.8

Now you can perform DNS lookups:
$ dig marathon.mesos
$ dig mesos-dns.marathon.mesos
$ dig master.mesos
$ dig slave.mesos

All of the above lookups will be answered by Mesos-DNS if the setup is correct.
###Example: Flask + Redis based page visit counter app
This is an atomicapp based on Nulecule specification. This is a 2-tier application based on Flask and redis. Every time you access the web page, a message is displayed which indicates the number of times the web page was accessed. And this counter is incremented upon each access.
To make this example work, make sure that Mesos-DNS is configured. The IP address of the server hosting Mesos-DNS needs to be the first nameserver entry in /etc/resolv.conf file. This makes it the primary DNS server. Related documentation link.
To be sure that Mesos-DNS is used as the primary DNS server, do following:
$ dig google.com

Here's the last few lines of the output I got. These are the lines of our interest:
;; Query time: 387 msec
;; SERVER: 10.2.2.2#53(10.2.2.2)      <------- Check this
;; WHEN: Tue Dec 29 07:01:52 EST 2015
;; MSG SIZE  rcvd: 55

If the response came from Mesos-DNS, the SERVER key will have the IP-Port combination of your Mesos-DNS server. If you don't get the expected results, try starting Mesos-DNS through command line (without Marathon) and see if it reports and warnings/errors.
With that setup in place, you can simply do:
$ atomicapp run --provider=marathon dharmit/flask_redis

Above command will download three Docker images. Each more than 250 MB in size. So depending on the Internet connection you have, above command will take some time to show the results.
Once the images are downloaded, you can check Marathon UI for the port on which Flask application is listening. Access that port from the host or vagrant box to check the page hit counter.