CI Example

gistfile1.md

CI Example

This is a very simplified and hypothetical example of a continuous integration / delivery system.

Major components

• Apache Mesos cluster configured with a master/slave setup.
• Marathon Mesosphere's Marathon framework running on Apache Mesos.
• Buildbot CI service with running in Docker containers.
• Docker Docker configured as a Mesos slave containerizer.

Definitions

• Project: A software project that is checked a VCS service like Github.
• Event: Anytime a meaningful action is triggered within a project repository, e.g. commit to master.
• Build: A project is compiled and built successfully into a Docker container and pushed to a private Docker registry.
• Deployment: A new version of software is deployed into a Mesos cluster.
• Endpoint: An endpoint is the external point of entry from the internet e.g. api.example.com:8080.

Workflow

Anytime an event occurs in a project a build is triggered by Buildbot and if it is successful Buildbot will schedule a deployment into an Apache Mesos cluster via HTTP POST request to Marathon. Marathon will then handle the graceful rollout of the service into the production cluster.

Project Configuration

Each project must provide a standard interface for Buildbot to detect and build consistently.

Project Layout:

Dockerfile
README.md
cd-example
setup.py
test

Each time a repository event occurs Buildbot will invoke a fixed set of buildsteps against a given project.

Example steps Buildbot might invoke:

> python setup.py test
> python setup.py sdist
> project=$project:$build-$branch-$(git describe)
> docker build -t registry.example.com/$project
> docker push registry.example.com/$project

If projects vary in language or configuration Buildbot can be configured to detect a "project type" and run different commands e.g. mvn package for Java, etc.

Build Versioning

An example Docker tag might look like below:

webApp-123-master-0.5.20-12-gf5615cf
# webApp - Name of the project
# 123 - The number of times this project has been built starting from 1
# master - The Git branch
# 0.5.20-12-gf5615cf - The output of Git describe.

If a branch is tagged "0.5.20" and we are currently at the revision that was tagged git-describe will output "0.5.20", however if we are beyond that revision it will show the number of commits you are ahead of the last tag and the beginning of the sha hash you are currently on. This is considered an unstable build.

Deployment to Marathon

Once a successful build has completed and an image has been pushed to the Docker registry a JSON document will be generated specifying that build and be PUT to Marathon's deployment interface. This will trigger an update of the existing application configuration and deployment of new containers gracefully into the Apache Mesos cluster.

Here is an example JSON document that can be PUT to Marathon's application interface. From here Marathon will handle the rest of the deployment

{
    "constraints": [
        [
            "hostname",
            "LIKE",
            "compute-slave-[4-6].example.com"
        ],
        [
            "hostname",
            "UNIQUE"
        ]
    ],
    "container": {
        "docker": {
            "image": "registry.example.com/$project_tag",
            "network": "BRIDGE",
            "portMappings": [
                { "containerPort": 31017, "hostPort": 0, "servicePort": 20000, "protocol": "tcp" }
      ]
        },
        "type": "DOCKER",
    },
    "cpus": 0.5,
    "healthChecks": [
        {
            "gracePeriodSeconds": 5,
            "intervalSeconds": 20,
            "maxConsecutiveFailures": 3,
            "path": "/",
            "portIndex": 0,
            "protocol": "TCP"
        }
    ],
    "ports": [ 31017 ],
    "id": "webApp",
    "args": ["example --argument 1"],
    "instances": 3,
    "mem": 512.0,
    "uris": [
        "/root/.dockercfg"
    ]
}

> curl -X PUT example-host:8080/v2/apps/webApp -d@marathon.json

Service Discovery

Once the application is deployed Marathon will provide a REST interface which will provide the dynamically allocated host ports which map to a statically defined "servicePort", this service port is used to proxy traffic to the dynamically assigned "hostPort".

> curl -X GET http://127.0.0.1:8080/v2/tasks
webApp 20000 compute-slave-4.example.com:31156 compute-slave-5.example.com:31153 compute-slave-6.example.com:31145

You can write your own solution to proxy to these dynamic ports or use an existing script provided with Marathon to load balance to HAProxy.