I recently got the opportunity to kickstart a project from the ground up. I didn't want to overload my non-technical client with recurring billing for services he wouldn't need. I ended up taking the time to implement a CI/CD environment that deploys to Heroku for free. My client only has to pay for his production Heroku dyno, the rest of the build pipeline is either free or, in the case of the staging environment, running on the free tier.
While I was about to start a new project this weekend, I decided to take a moment to document my process to help myself and you with future projects. I'll outline some of my favorite practices for deployment environments and why I follow them. In this article we'll be hosting the code for a Ruby on Rails application I'm building, for a personal project, at GitLab.com. We'll be using their shared runners for CI/CD and deploying to Heroku in staging and production. As an added bonus, we'll use Slack integrations to provide us an eagle eye view of the whole operation.
I'd like to get this out of the way first. I'll be using GitLab and Heroku because of their convenience and low cost. I chose GitLab because of their free private repositories. I wanted to avoid charging my client for services he wouldn't need or understand, and GitLab's integrated free CI/CD services led me to avoid paying another vendor.
You can very well use a combination of other services to achieve the same goal, and I could have used Github to host my code, Codeship or Jenkins for CI/CD and a private Kubernetes cluster running on AWS for staging, deployment and my CI/CD runners, but the project didn't require such an elaborate setup and I didn't want to spend more than a day on building the pipeline.
There are many explanations out there for CI/CD, and being self-taught it took me a while to fully understand what it was and what it implied. Here's my definition:
CI/CD is a predictable, repeatable, and automated process that integrates new code seamlessly, converts it into a final product, and optionally publishes it to production.
Keywords:
- Predictable: you can predict the outcome of the pipeline given the input. Bad code will result in the pipeline halting, while good should breeze through.
- Repeatable: the process is remains constant and is repeated every time it is needed. This becomes easier with the next keyword:
- Automated: the entire process is run by a computer every time it is needed, with little or no human intervention
- Process: CI/CD isn't a tool, it's a process. You can build a CI/CD process using any number of tools on the market, but the process is not the tool.
Since we're deploying a Rails app, this article will be tuned to web applications, but the core concepts can be applied to any type of project with a few changes. When I began programming, I was already always using a form of CI/CD since my first real application used Github to auto-deploy my code to Heroku, but since then I've started using a robust CI/CD pipeline to automate everything including tests and deploys to staging.
I like to break my CI/CD pipeline can be broken into three components:
- Continuous Integration
- Continuous Delivery
- Continuous Deployment
Continuous Integration (CI) is the practice of merging all developer working copies to a shared mainline several times a day -- Wikipedia
In English, CI is ensuring that everyone has up to date access to the codebase at all times. Achieving this can be as simple as using a central code repository like Github or GitLab or your favorite SCM service. As long as you have a central repository where developers can merge their code frequently, give them a good criteria on the kind of code you want, and you're good to go.
However, a simple code repository isn't a good CI solution. A robust CI process shouldn't even let you merge bad code. A well made CI process should test every code proposal and give developers feedback on why their code might not be up to par.
On a project with multiple contributors, it's also great to place a moderation system before merging code. Even setting a minimum number of LGTM from fellow contributors should result in a big difference.
Continuous delivery (CD) is a software engineering approach in which teams produce software in short cycles, ensuring that the software can be reliably released at any time -- Wikipedia
With a good CI process you should always have a fully tested repository containing the latest version of code. CD takes you a step further by automating the build process too! By building a CD process on top of CI, we can ensure that we always have a production ready product with the latest code that is always ready to ship!
At this point, you can take your process a bit further and add Continuous Deployment. Since you already have a final product ready to ship, you should considering automating the deploy process too so that going "live" is just as easy as being "ready"
I feel that the best way to describe what we'll be achieving is through features. Let's discuss our CI setup using Gherkin. What we're trying to achieve is this:
Feature: Continuous Integration
As a developer I should get instant feedback on my code proposals Because this will keep me from merging bad code And provide me feedback on how to fix my code
Before we run through our scenarios, let's add some context
Background:
Given I am a developer on the project And I have the ability to make pull/merge requests
Let's build a scenario for a good code proposal
Scenario: I open a merge request with good code
Given I have written good code When I open a merge request Then A CI pipeline should start And The pipeline should pass And I should get feedback that my code passes
Let's build a scenario for a bad code proposal
Given I have written bad code When I open a merge request Then A CI pipeline should start And The pipeline should fail And I should get feedback on why my code fails
Now that we have an idea of what we want to achieve, let's talk about how we're going to achieve this. The primary difference in both our scenarios is whether our code is good or bad, but what is good code?
The requirements for what makes code good or bad depends on your project, but it's important that your process of deeming code good is clear. In my project, I plan to have four specific requirements. The code:
- should be in line with my project's goals
- should follow my personal style of Ruby code
- must pass all acceptance tests and unit tests
Since we'll be using Gitlab CI for our projects, i'm going to be creating a gitlab-ci.yml file and describing the steps in my project's testing pipeline. Since GitLab CI supports specifying custom docker images, i'll be using the ruby:2.3 image for my tests. I do this by defining the first line as so:
# gitlab-ci.yml
image: ruby:2.3
Since there's no way to automate whether code is in-line with my project's goals I'll be formalizing my application's goals in my README. If you have multiple contributors to the project, you can further strengthen this by requiring a minimum number of LGTM's before merging a request.
I can't publish my project's specifications because they're not ready yet for public review, but here's a curated list of awesome README files. They all have a clear description of what the project does, for a new project, you'll be defining what the project will do.
I formalized my personal style of Ruby code a while ago by defining a .rubocop.yml for myself a while ago. If you don't use linters, they're a great way to enforce syntactical correctness of your codebase. They test not just whether your code is valid, but you can even formalize conventions such as using two spaces for indentation, etc. Here's a list of code linters that you can use with your favorite language.
Testing with Rubocop is straightforward, so I'll just add the script to my gitlab-ci.yml file:
rubocop:
type: test
script:
- bundle install --path /cache
- rubocop
Since i'll be using Cucumber for acceptance tests, and Rspec for integration and unit tests, we'll have to run these tests separately.
cucumber:
type: test
services:
- postgres
variables:
DATABASE_URL: "postgres://postgres@postgres/myapp"
script:
- mkdir /tmp/phantomjs
- curl -L https://bitbucket.org/ariya/phantomjs/downloads/phantomjs-2.1.1-linux-x86_64.tar.bz2 | tar -xj --strip-components=1 -C /tmp/phantomjs
- mv /tmp/phantomjs/bin/phantomjs /usr/local/bin
- apt-get update -qy
- apt-get install -y nodejs
- bundle install --path /cache
- bundle exec rake db:setup RAILS_ENV=test
- bundle exec rails cucumber
rspec:
type: test
services:
- postgres
variables:
DATABASE_URL: "postgres://postgres@postgres/myapp"
script:
- apt-get update -qy
- apt-get install -y nodejs
- bundle install --path /cache
- bundle exec rake db:setup RAILS_ENV=test
- bundle exec rake spec