I love working at Constant Contact. But like any engineering group, technology can easily lag over time, drift between teams, and generally become a hassle. This time last year I wanted to do something about it, specifically for front-end tooling (since that's what I work with every day).
Your team trying to keep up with tech debt
After talking with some fellow engineers, we identified some key issues:
- How do we keep frontend tech up-to-date? We're not talking about hopping on the latest framework hotness. We mean tools and techniques that can simplify writing and shipping JavaScript - ES2015+, Babel, Webpack, modular code, etc.
- How do we bridge the gap between teams? We're pretty good at coordinating work across teams. But our build tools became out-of-date and ownerless, so no team had stepped up to fix them in a few years.
- How do we do this with as few resources as possible? We were still adjusting from some major company changes last year. So having the engineering group drop everything to support a tooling moderinzation wasn't gonna happen.
And so I started a grass-roots group of engineers who wanted to improve our situation. We called it the "Frontend Guild." Our mission statement:
The Frontend Guild is a group of engineers who want to push for Stability, Maintainability, and Modernization on front-end projects across the organization. We will do this by providing shared tools, patterns, and resources for teams across the company. Together we will make tech enablement a dedicated effort at Constant Contact.
Leading and organizing this group has been an incredible learning experience. Here's the short version of how I ran the group, the successes we had, and the challenges we faced.
The first thing we did is come up with a survey for all teams to fill out. We asked ~25 questions about the state of their applications and tooling - library versions (jQuery, underscore, etc), Node versions (for tooling), ES6 or ES5 - as well as opinions such as "how happy are you with your build setup" or "do you find your UI automation reliable". We got about 20 responses, representing most apps created over the last few years. Even if the Guild idea were to fall apart here we would have done something useful - in taking a rough inventory of technology across the company, we can provide awareness for both engineering and management.
We learned a lot, very quickly
We followed this up by asking each team to give a presentation about their projects to the Guild to get some more in-depth info than the survey. Each team was represented and so awareness of the Guild spread, and we kept the meetings bi-weekly so as not to overwhelm anyone. Attendance was good in these early days since everyone came out to support their team's speaker. This rounded out our knowledge from the survey and got us ready for the next exciting phase...
In the first month...
| Successes | Challenges |
|---|---|
| Described the state of frontend tech at the company | None yet! |
| Started holding regular meetings | |
| Spread awareness of the Guild |
OK, no one really finds backlog grooming exciting. Our meeting attendance began to slip pretty heavily, from a rotating group of 20+ people to a core group of around 8. But we had sufficient cross-team representation that one team wouldn't be making decisions for the whole company (and lead to unnecessary fights). And someone had to turn all our gathered knowledge into something useful!
Backlog grooming is rarely this pleasant
After pouring over the survey we came up with a bunch of high-level "topics" such as:
- Enable ES6, Sass, ESLint, ESDoc, etc: Upgrade our build tooling to enable the latest tech for developing and testing.
- Webpack and "Modularize" Code: Enable proper dependency resolution, simplify code bundling, and enable the development of smaller, sharable code modules.
- App/Component Generators: Allow developers to create new projects - and upgrade existing ones! - using a command-line tool.
Potential topics for the Guild to tackle
Using these topics as a guide we began form them into epics, each with several specific implementation stories. A pattern began to emerge - demo a build tool, integrate it into a new project, test it in an existing project, and wait for adoption. One tool at a time, it seemed like we could slowly bring about a large number of improvements. But moving slowly doesn't get developers excited, and it certainly doesn't improve meeting attendance.
In the first three months...
| Successes | Challenges |
|---|---|
| Over a dozen high-level "epics" | Reduced meeting attendance |
| Several epics groomed into a few dozen workable stories | No tangible progress, people getting antsy |
And so a backlog was forming but we had no one to do the work! We considered asking for a full-time engineer to work on it, but that idea got no traction this early on. And I didn't want to rely on volunteering alone - when push comes to shove sprint work will always come first, leaving Guild work to fall to the wayside.
We discussed this predicament during Guild meetings, and we came up with a compromise - a rotating "Task Force" consisting of two engineers from different teams for one month at a time. This would provide a constant work force without sacrificing too much from their primary work. By pairing, they would be able to solve problems together. And if they come from different teams, knowledge from the Guild would be able to spread to more teams more quickly.
How we hoped Task Force members would work together
This plan quickly fell apart. We started with a short list of interested engineers, enough to last a few months. But some engineers felt uncomfortable leaving their teams to make up for them during a busy sprint. Others simply lost interest and wanted to focus on their own work. After some lineup changes we only ever got one pair of engineers together, but we did get four back-to-back months of volunteers, most one engineer at a time.
Those of us not actively volunteering helped by reviewing code. We required two reviewers - both from different teams - to approve a pull request before merging. This was to ensure a wider variety of opinion, since we've found teammates tend to think somewhat alike (for better or worse).
Our first engineer on the Task Force made the smart choice to focus on the "App Generator" work first. This would be the vehicle for all other tooling changes. And so he made a custom Yeoman generator for scaffolding new JavaScript projects, both "apps" (single-page applications) and "libraries" (smaller, sharable components). He set it up to create new projects built using Webpack with plugins for Babel, Sass, EJS templates, and more. We continued to use Grunt to tie together all our tools and commands, including Webpack.
CTCT <3's Yeoman
The next few Task Force engineers built upon the initial progress - unit testing (with code coverage) via Karma, UI automation testing with Protractor, a sample Marionette app, and more. Most importantly, we built in the ability to "extend" core config files - package.json, .eslintrc, and Webpack configs to name a few. This would enable us to deliver a "base" config that could change over time while maintaining the ability for teams to provide overrides to those files.
# "base" config lives here, should be overwritten by the generator
config
grunt/
aliases.js
...
webpack/
webpack.dev.config.js
webpack.prod.config.js
.eslintrc.js
karma.conf.js
# Overrides go in top-level files, won't get overwritten
grunt/
aliases.js
...
# All source code - JS, CSS, HTML, etc
src/
# All unit tests
tests/
# "top-level" config files, also for overrides
.eslintrc.js
karma.conf.js
webpack.dev.config.js
webpack.prod.config.jsSome of the directory structure produced by the generator
In order further spread the word about our work, I had begun sending out monthly emails (made with Constant Contact, of course!) describing our progress on the Generator and other news from our meetings. Beyond information, I encouraged people to stay involved by coming to Guild meetings. We also used our HipChat room as a place for tech support, which our regular members graciously helped provide.
After about seven months...
| Successes | Challenges |
|---|---|
| A temporary but successful Task Force to get work done | Less Task Force engagement than we anticipated |
| A working Yeoman generator for creating and upgrading JavaScript apps | Only a few teams contributing directly |
| Email updates to keep the organization informed of our progress |
We had run out of willing volunteers, and a few weeks went by with no active Task Force. I had been running meetings and doing code review thus far, so I stepped up to do active development on the Generator for a short time. A few final PRs later and we were ready to show off work!
Our developer reserves had run dry
Along with the Task Force volunteers, we presented our progress to the company over several engineering meetings. This and a little in-person badgering led a few teams to start using the Generator. Early reviews were positive, especially in regards to daily development - with the Webpack dev server, changes built nearly instantly and your local page would auto-reload!
Now that teams were beginning to use our tool, our meetings had become weekly. Though attendance remained fairly regular at 6-10 people, this helped us keep up with bug reports and remain as open to the organization as possible. For a while we would have to find time to fix these bugs in between our own sprint work. But it was necessary and our managers supported us when it became a priority over our normal work.
By this point we had proven to the organization the value of the Frontend Guild and the Generator. And so we finally got what we needed - a full-time engineering position! One of the past Task Force volunteers, a relatively recent hire, transitioned into the new role. Embedded in our Continuous Delivery team he began to integrate the Generator more closely with our existing Jenkins-based tooling for building and deploying JavaScript apps, in addition to more standard feature development.
After about a year...
| Successes | Challenges |
|---|---|
| Several teams using our tools to create new apps and update them on a regular basis | More pressure to keep things working |
| A permanent engineer working on the Frontend Guild backlog | Cutting into regular work to keep up with demand |
The Generator is out in the wild but our work is far from over. Of course there are many more improvements to make, especially in ensuring the quality of the Generator itself. We primarily test by generating new apps, so identifying issues when upgrading from any old version to the latest is tricky. We're looking into doing more manual regression testing before each release to try and catch more bugs before our users do. If the manual process helps, we'll try to automate the process to speed things up.
As more teams adopt the Generator - and therefore Webpack and modular JavaScript - we'll begin testing the limits of our approach. For each individual project, how do we best ensure that they rerun the generator regularly to stay up to date? As projects become intertwined, how will we deal with dependency conflicts? How will we properly split up our testing? How will these new apps play with our legacy projects? We have some answers and guesses for now, and we'll keep working towards the best solutions.
Whatever changes may come there will be a lot of work coming up. After only two months we're seeing that a single full-time engineer might not be enough to keep up with bug fixes, features, and tech support. Though the "Task Force" never officially ended, finding new volunteers has been a fruitless effort after the first group that stepped up. We're starting to discuss whether to formalize the program - have managers assign engineers every so often - or hire another engineer, but it's too soon to tell what we'll do exactly.
We know our immediate future lies in the Generator, but I also want to get an idea of what might come after. And so we've also begun a second round of knowledge gathering. Instead of getting every single team involved, we're targeting project leads and critical apps/services. Our goal this time is to identify the next "big thing" we can tackle - after the Generator, what other ways can we improve the frontend development experience at Constant Contact?
I certainly put a lot of work into the Frontend Guild: hosting meetings, taking notes, planning work, finding volunteers, giving presentations, reviewing pull requests, contributing code... And all of that in between normal sprint work, too! But I was far from going it alone. I had support from my manager on how best to organize the Guild. I received technical guidance from senior engineers on the goals of the tooling and in seeing the big picture. And if it wasn't for my generous coworkers who come to Guild meetings, volunteer for the Task Force, review code, and more, all of this would never have happened!
But all that hard work has paid off. I'm seeing the benefits of our work every time I work on my project. Most other teams have one or more projects using the Generator as well. Now that most teams have the same, updated base setup we can more easily help each other out. And when we solve a problem for one project, we're really enabling all teams at once. Not only has tackling tech debt become a serious focus for us, we're dealing it faster and better than ever before.