Software deployment. It seems like such an easy concept. I wrote some new code and now I want to get it into the hands of my users. But there are few areas in the world of software development where you find a more diverse set of approaches to a such a simple-sounding problem. In all my years in the software business, I don't think I've ever seen two deployment processses that are the same. So many different tools. So many different approaches. It turns out it is a pretty complicated problem with a lot of moving parts.
But there is one over-arching trend in the world of software deployment that seems to have almost universal appeal. More. More deployments. More often.
Ten years ago it was common for a software deployment to happen a few times a year. Software changes would be batched up for weeks or months waiting for a release cycle and once the release process started, development stopped. All attention was focused on finding and fixing bugs and, eventually, releasing the code. It was very much a bimodal process: develop for a while and then release for a while.
Now the goal is to greatly shorten the time it takes to get a code change deployed, to make the software deployment process quick and easy. And the best way to get good at something is to do it a lot of times.
"Repetition is the mother of skill." - Anthony Robbins
If we force ourselves to do software deployment frequently and repeatedly we will get better and better at it. The process I use to put up holiday lights is appallingly inefficient and cumbersome. But since I only do it once a year, I put up with it. If I had to put those lights up once a month or once a week or once a day, the process would get better in a hurry.
The ultimate goal is Continuous Deployment, a continuous pipeline where each change we commit to our VCS is pushed through a process of testing and then, if the tests succeed, is automatically released to production. This may be an aspirational goal for most people and there may be good reasons not to have a completely automated pipeline (e.g. dependencies on other systems) but the clear trend is towards frequent, repeatable software deployment without the bimodal nature of traditional deployment techniques.
Which brings us to the real topic of today's post. AWS CodeDeploy is a new service from AWS specifically designed to automate code deployment and eliminate manual operations.
This post will not be a tutorial on how to use AWS CodeDeploy. There is an excellent hands-on sample deployment available from AWS. What this post will focus on is some of the specific features provided by AWS CodeDeploy that might help you achieve the goal of faster and more automated software deployments.
This may seem like a contradiction given that this is a new service from AWS but the underlying technology in AWS CodeDeploy is not new at all. This is a productization of an internal system called Apollo that has been used for software deployments within Amazon and AWS for many years.
Anyone who has worked at Amazon will be familiar with Apollo and will probably rave about it. Its rock solid and has been used to deploy thousands of changes a day across huge fleets of servers within Amazon.
You can control how AWS CodeDeploy will roll out the deployment to your
fleet using a deployment configuration. There are three built-in
configurations:
-
All At Once - Deploy the new revision to all instances in the deployment group at once. This is probably not a good idea unless you have a small fleet or you have very good acceptance tests for your new code.
-
Half At A Time - Deploy the new revision to half of the instances at once. If a certain number of those instances fail then fail the deployment.
-
One At A Time - Deploy the new revision to one instance at a time. If deployment to any instance fails, then fail the deployment.
You can also create custom deployment configurations if one of these models doesn't fit your situation.
If you are deploying your code to more than one instance and you are not currently using Auto Scaling you should stop reading this article right now and go figure out how to integrate it into your deployment strategy. In fact, even if you are only using one instance you should use Auto Scaling. Its a great service that can save you money and allow you to scale with demand.
Assuming that you are using Auto Scaling, AWS CodeDeploy can integrate with your Auto Scaling groups. By using lifecycle hooks in Auto Scaling AWS CodeDeploy can automatically deploy the specified revision of your software on any new instances that Auto Scaling creates in your group.
AWS CodeDeploy uses a YAML-format AppSpec file to drive the deployment
process on each instance. This file allows you to map source files in
the deployment package to their destination on the instance. It
also allows a variety of hooks to be run at various times in the
process such as:
- Before Installation
- After Installation
- When the previous version of your Application Stops
- When the Application Starts
- After the service has been Validated
These hooks can be arbitrary executables such as BASH scripts or Python scripts and can do pretty much anything you need them to do.
Below is an example AppSpec file.
os: linux
files:
- source: Config/config.txt
destination: webapps/Config
- source: source
destination: /webapps/myApp
hooks:
BeforeInstall:
- location: Scripts/UnzipResourceBundle.sh
- location: Scripts/UnzipDataBundle.sh
AfterInstall:
- location: Scripts/RunResourceTests.sh
timeout: 180
ApplicationStart:
- location: Scripts/RunFunctionalTests.sh
timeout: 3600
ValidateService:
- location: Scripts/MonitorService.sh
timeout: 3600
runas: codedeployuser
AWS CodeDeploy can be driven either from the AWS Web Console or from the AWS CLI. In general, my feeling is that GUI interfaces are great for monitoring and other read-only functions but for command and control I strongly prefer CLI's and scripts so its great that you can control every aspect of AWS CodeDeploy via the AWSCLI or any of the AWS SDK's. I will say that the Web GUI for AWS CodeDeploy is quite well done and provides a really nice view of what is happening during a deployment.
There is no extra charge for using AWS CodeDeploy. You obviously pay for all of the EC2 instances you are using just as you do now but you don't have to pay anything extra to use AWS CodeDeploy.
The previous section highlights some features of AWS CodeDeploy that I think could be particularly interesting to people considering a new deployment tool.
In this section, I want to mention a couple of caveats. These are not really problems but just things you want to be aware of in evaluating AWS CodeDeploy.
AWS CodeDeploy only supports deployments on EC2 instances at this time.
AWS CodeDeploy requires an agent to be installed on any EC2 instance that it will be deploying code to. Currently, they support Amazon Linux, Ubuntu, and Windows Server.
Because of the way AWS CodeDeploy works, there really isn't a true rollback capability. You can't deploy code to half of your fleet and then undeploy that latest revision. You can simulate a rollback by simply creating a new deployment of your previous version of software but there is no Green/Blue type rollback available.
We just created a new deployment pipeline at work that implements a type of BLUE/GREEN deployment and is based on Golden AMI's. We are very happy with that and I don't think we will be revisiting that anytime soon. However if I was starting that project today, I would certainly give a lot of thought to using AWS CodeDeploy. It has a nice feature set, can be easily integrated into most environmenets and code bases, and is based on rock-solid, proven technology. And the price is right!
Links: