Mobmerry_CI_Setup.md

Mobmerry CI Setup

---

Tech Stack:

1. Docker Engine for Linux Containers
2. TeamCity as CI Server

---

Config Files and Scripts

1. Dockerfile creates container image with all the infrastructure required for Mobmerry
   • Ruby
   • MongoDB
   • Elasticsearch
   • Redis
   • Nodejs
2. docker_scripts folder contains scripts required to configure and start the container:
   • test_entrypoint.sh runs when container starts, it starts all services and does some prerequesite work to run tests.
   • wait_for_it.sh (from this github repo) waits for all services to be up and listening.

---

Building Mobmerry Image Container

Build the image using:

docker build -t interactionone/mobmerry:v1 .

Here interactionone/mobmerry is the name for image, you can give any name.

And v1 is the tag / version.

-t flag will attach a pseudo tty to the image building process to see the output

The above command will use any cached image layers if available in the system.

If you want to start afresh and not use any cached image layers, use:

docker build -t --no-cache interactionone/mobmerry:v1 .

---

Starting and Running the Container

There are two modes you can run the container as facilitated by test_entrypoint.sh

Command Mode

In this mode, when you start a container with a command, the container executes that command and exits.

For Eg:

docker run -it --rm interactionone/mobmerry:v1 rspec spec/models

Here rspec spec/models is a command that runs against a new container that is started.

When the command exits, the container stops and exits.

Flags -

• -it will make container interactive (i) and attach a tty (t) to the container process running
• -rm will remove the stopped container as soon as process exits

To gain shell access to the container:

docker run -it --rm interactionone/mobmerry:v1 /bin/bash

This executes /bin/bash command against a new container that is started.

Standby Mode

You can start a container to run in a standby mode - as in no command is provided to execute immediately but gets container ready to accept any commands passed to it subsequently.

docker run -d --name mobmerry_app interactionone/mobmerry:v1

This will have the container named mobmerry_app started and waiting for executing any commands to be passed to it as such:

docker exec -it mobmerry_app rspec spec/models

The container will receive the command, execute it, go back to the standby mode.

Flags -

• -d will start the container in daemon mode

To get shell access to such idle container

docker exec -it mobmerry_app /bin/bash

To stop such a container

docker stop mobmerry_app

You can start many containers, either in standby or command mode - the number is only limited by system capabilities

---

Integrating with TeamCity CI Server

Install and start TeamCity by following instructions here

TO USE POSTGRES DB, read this guide from TC

By default TeamCity comes with a single Build Agent

Number of Build Agents determine how many parallel executions you can do in a build.

So we will install 2 additional build agents in the same machine - max allowed for free TeamCity license.

To Install Additional Build Agents:

• Go to Agents Tab in TeamCity web UI at http://localhost:8111
• On Install Build Agents menu on top right select zip file distribution
• Unpack the zip to a folder - and make 2 copies of the same folder - for 2 build agents
• In conf folder of each of the agents, rename buildAgent.dist.properties to buildAgent.properties
• In buildAgent.properties provide a different name= of each of the build agents
• Do bin/agent.sh start in both the agent folders to start the agents.
• This can be added to an init script, alongside the start procedure for TeamCity.

TeamCity Project Creation

Create a new CI user

Create a new project Mobmerry

In Project Settings add a VCS Root, use auto-detect from URL option, use github URL in Repository URL with username and password.

--> Create.

Edit the created VCS Root as some configuration options does not show up when creating it.

Click on Show advanced options at the bottom of the page, locate Branch Specification config field and add this

+:refs/heads/*
+:refs/pull/(*/merge)

• The first specification will pick up all branches in the github repo to run CI Build on
• The second specification picks up pull requests from github to run CI build on

Creating Build Configurations

Current setup we have 4 build configurations - better to call them as build steps

1. CI Setup

   This step will:

   • copy mobmerry config files to checked-out commit folder
   • build the mobmerry image using image layer caching and picking up only changed contents from checked-out commit - thus being very quick

2. CI Test Channel 1, 2 and 3

   • These steps will wait for CI Setup to end, then all of them start parallely in 3 agents configured.
   • They each start up a container built from CI Setup and pass a command to execute for eg: rspec spec/models

Now we will create these Build Configurations.

Go to Mobmerry's Project Settings locate Build Configurations and click Create build configuration

Create 3 configurations with the names as mentioned in steps above.

Edit each of the Build Configurations from Project Mobmerry's settings page

1. CI Setup

   Version Control Settings - Attach existing VCS root --> Select Mobmerry VCS Root

   Build Step - Add Build Step - Runner Type: Command Line -

   • Build Script -

     cp ~/mobmerry_config/* ./config/
     docker build -t interactionone/mobmerry:v1 .
     

   Triggers -

   New Trigger - VCS Trigger

   • Check Trigger a build on each check-in
   • Add a new Trigger --> Select Trigger Build, let other configs here be default values.
   • Click on Show advanced options --> in Branch filter, add rule -:*/merge, this will not trigger builds for any new pull requests that may come. Great video here to learn on how to set up VCS branch specifications everywhere

   Parameters -

   • Add env.DOCKER_HOST = tcp://localhost:4243

2. Test Channel 1, 2 and 3

   Build Step - Add Build Step - Runner Type: Command Line -

   • Build Script -

     Add different rspec folders to test in each of the Channels

     As:

     docker run --rm interactionone/mobmerry:v1 bundle exec rspec spec/models
     

   Triggers -

   • Wait for CI Setup to finish in all of the channels to parallely trigger off all channels

   Parameters -

   • Add env.DOCKER_HOST = tcp://localhost:4243

You can also manually run builds by going to build configurations and clicking run button. But this will not trigger the dependent child builds.

---

Gotchas

• In teamcity's build agents you cannot access docker for some reason even though agent runs on same user

  So this needs to be done :

  in /etc/default/docker file
  add DOCKER_OPTS="-H tcp://localhost:4243 -H unix:///var/run/docker.sock"
  save and close file
  
  restart docker service
  
  Add environment variable DOCKER_HOST=tcp://localhost:4243