Although I work for an AWS partner, I've always shied away from AWS CloudFormation, their orchestration technology. However, when I got talking to Will at LocalGovDrupal about reducing the barrier to entry for smaller organisations who want to use the distribution as a starting point, a plan formed. Perhaps the AWS Marketplace would be a good way to allow other councils to find and try out LocalGovDrupal?
The kicker of course is to put something in AWS Marketplace you pretty much need to build it with AWS CloudFormation. CloudFormation basically provides a YAML or JSON formatted means of describing any AWS service so it can be automatically set up and configured, and as a result your infrastructure can be saved in code, in much the same way as the popular Terraform from Hashicorp does, not to mention the AWS Ansible modules. After some Googling I fell upon the AWS Reference Architecture project for Highly Available Drupal. It was pretty out of date, but nice and complete, even with DNS handling and SSL and CDN configuration right out of the box. It has issues though:
- It's really old! So it's using an old version of Drupal, an old version Amazon Linux, even an old PHP (7.0) so all that needed updating
- It wasn't actually making any effort to handle installing Drupal, in spite of build parameters that implied it would
- LocalGovDrupal installs with
composer, but the providedinstall_drupalscript assumed it could just unzip a downloaded archive - All the instance types were woefully out of date
- It didn't work in AWS regions that weren't yet supporting all the necessary products when it was made (including London, crucially)
So more work that I intended to do - I could've just packed an Amazon Machine Image (AMI) and had done with it, but I decided I wanted to bring this reference architecture into 2021 so I could learn more about CloudFormation and also create a potentially useful "production release" Marketplace product. All the changes I had to make are recorded her for posterity - codeenigma/aws-refarch-drupal#1
But in essence, here's how it works:
The master template defines all the parameters CloudFormation needs the end customer to enter in order to build the infrastructure and install Drupal. Then for each requirement element it loads in a Resource sub-template that does the actual creation of that element. The order is set using DependsOn in the master template, so because the 'newvpc' resource has no dependencies it naturally gets built first, everything else depends on it because it builds a new Virtual Private Cloud (VPC) - a standalone virtual network at AWS. Once that is built and our new network is in place, it moves on to the 'securitygroups' resource, which orchestrates the creation of the firewall rules separating the various bits of infrastructure from each other and the outside world.
After that, there's a whole bunch of infra we can build at once, because it all DependsOn the VPC and the Security Groups (SGs) existing. So next comes (in no particular order, CloudFormation can build them asynchronously) the:
- 'bastion' resource - an AWS AutoScaling Group for EC2 (ASG) for creating a server to use as a jumping off point to SSH to the web server cluster, if needed
- 'efs' resource - for mountable network disk
- 'publicalb' resource - to create our load balancer which will sit in front of our web servers
There are a few other resources that depend on the VPC and the SGs, but which also have their own special extra dependencies, namely the:
- 'elasticache' resource - this runs up an ElastiCache cluster for Redis or memcached, but it has an extra dependency - it's optional on the user entry form, so if it isn't asked for it isn't built
- 'rdscluster' resource - creates an AWS Aurora (their MySQL flavour) database cluster - the only option in the reference architecture, but I altered this to offer the choice of an:
- 'rdsinstance' resource - which creates a standalone, single or multiple availability zone MariaDB instance, a single AZ instance being far cheaper to operate than a full Aurora cluster!
Once that stuff is all up, CloudFormation can create the ASG for the web servers. This is similar to the 'bastion' resource, except it has a few extras. It needs to:
- install PHP and dependencies for LocalGovDrupal
- install composer
- install Apache
- install Drupal using
composeranddrush si
This all happens via a cloud-init script - https://cloudinit.readthedocs.io/en/latest/ - which gets described in the YAML file of the 'web' template and is built and executed by the wrapper AWS have created call cfn-init (cfn being shorthand for CloudFormation).
Once that's done, it's all over bar the shouting. You might have to wait 15 minutes or so to actually see Drupal, because the composer install and the drush si take quite a bit of time, but if you login to your AWS console and go to EC2 and Load Balancers you'll see your new load balancer, and there should be a Drupal website on its URL.
There are just two more resources in the master template we haven't loaded in yet, 'cloudfront' and 'route53'. These are both options on the form, but if selected, once the 'web' template has successfully built its ASG then these templates will be executed, one creates a DNS record for the new Drupal website (only if you're using AWS Route 53 for DNS services, of course) and the other puts the CloudFront CDN in front of your load balancer, for better performance, SSL as standard and DDoS protection (which I recommend - it's not even expensive).
And that's the stack! Once it's all up, you'll find a vanilla LocalGovDrupal waiting for you to play with it.
Is it in the Marketplace? No, not yet - we're not sure how/who should do this, so it needs to be discussed. Can you use it? Yes! The good news is all your need is an active AWS account and you can have at it! Just go to this project on GitHub and click on Launch Stack next to the region you want to use, fill in the form and off it will go! https://github.com/codeenigma/aws-refarch-drupal/tree/master#running-localgovdrupal-on-aws
Any issues, let me know via the GitHub issue queue for the project.