Chef Gemfile.lock FAQ

gemfile.lock.md

Gemfile.lock FAQ

1. What's the Gemfile.lock in chef for?

   chef uses a Gemfile and gemspec to control the ruby components we bring into Chef and the ChefDK. These files specify constraints such as "chef MUST have test-kitchen version 1.7 or greater!" but do not specify exact versions, so when new copies of dependencies (like test-kitchen) come out, we can pick them up without changing the Gemfile. The Gemfile.lock is a set of explicit versions ("Use version 1.7.1 of test-kitchen") generated by running the Gemfile one day.

   This is important because it locks down the set of dependencies we tested at, so that we have a "last known good" build. More importantly, it means that we can update our dependencies deliberately and safely through a Pull Request that goes through all testing. This means that master only ever gets a Gemfile.lock that has actually passed the tests.

2. Why put a Gemfile.lock in chef?

   In short, because we want to keep "interrupt work" to a minimum so that our talented Chef engineers can produce more features and bugfixes more quickly. Dependency updates--both for internal, Chef-produced components, and external components like the json library--are a major source of instability in master. Right now, when master is red, we treat it as a highest-priority interrupt--and interrupts have a peculiar quality which decimates our ability to plan and bring new features and bugfixes to Chef.

3. Why is master being red such a big deal anyway?

   If master can't build or fails its tests, our whole quality pipeline stops. Chef source code doesn't live in a vacuum: it is part of a pipeline where packages are produced, lots of tests are being run, and it is being dogfooded internally and externally. It's absolutely crucial to the quality of Chef that this pipeline remain operational.

   When master is broken for any reason, it causes a lot of bits of we treat it as a highest-priority interrupt because it stops anyone from being able to test their PRs and dogfooding the product.

4. Isn't dependency breakage a high priority though?

   Yes! Our dependency updater is a pipeline that we have to keep moving. It brings in new versions of not just external features, but also closer-to-home dependencies we want to bump frequently such as ohai, inspec, test-kitchen and berkshelf. The change here is that if a dependency break happens at 5PM on Friday, we probably won't look until Monday :)

5. How quickly will you fix it if a newer dependency does break chef?

   As soon as we discover that the PR to update the dependencies fails, we file a high-priority bug. The key for us is, this is high priority but not omg the world is broken priority--work on current issues can continue, master will stay green, and we will pick up the dependency update bug as soon as we finish current in-flight work.

6. If you're locking deps, how will you keep everything up to date?

   The biggest trap some projects get into with Gemfile.lock is that it never gets updated. we're committed to updating often, and have even written a bot that will check for new dependencies periodically (every 15 minutes currently). If the new dependencies pass all our tests, the Gemfile.lock will be updated in master and new releases will be built containing them. If the tests fail, a bug will be opened and we will investigate.

7. I use Chef as a a gem. Will this break me?

   Because our dependency versions are updated frequently, and because we will have a release-blocking bug if they fail, released Chef gems will always be safe against the latest current dependencies when it comes out. We will not release unless the dependency updater is green.

8. I use master of Chef in a Gemfile. Will this break me?

   If dependencies break and you're using master of Chef in your own Gemfile, it's possible for you to see the breaks (just like happens now). Since it could take a day or two before we fix it, you could be broken that long. Fact of life.

   If that time period of breakage doesn't work for you (and I understand!), there are ways to consume Chef's entire Gemfile.lock from master so that you are only using the latest tested dependencies. There is sample code in chef to do this (TODO I will post this--@jkeiser) and it won't prevent you from adding extra dependencies (such as your own app).