So, I didn't think of any of this until just about the time that we released the environments feature, which was way too late to do anything about it. Anyway, here's my alternate conception of environments.
- Cookbooks always get their version from a checksum of their contents. The way the checksum is generated isn't particularly important, but it's probably a checksum of all the individual files' checksums, like a Merkle tree without the tree.
knife cookbook upload
always uploads to a particular environment. The command line invocation can be the same as it is now, or more git-like, e.g.,knife cookbook upload ENV COOKBOOK
. The "_default" environment is assumed if no environment is given.knife cookbook upload
always constrains the environment to that exact version of the cookbook.
- You can't have two active versions of a cookbook in a single environment. If cookbook A needs cookbook Z at version 1 and cookbook B needs cookbook Z at version 2, and you need both A and B to run your infrastructure, then you need to modify one or more of the cookbooks to fix the incompatibility.
- The x.y.z version field of cookbook data may no longer have meaning to the chef-server, as environments would only care about the cookbook's checksum version. See next bullet point for discussion of how version constraints might be handled.
- Versioned dependencies in cookbook metadata must either be ignored, enforced by chef-client after fetching cookbook updates, or verified by the server upon upload. Each of these approaches has unfavorable trade-offs. Enforcing version constraints on the client side adds considerable delay between when a user causes and error and when they detect it; enforcing version constraints on the server may make it annoying to upload multiple cookbooks with "interlocking" version constraints. Ignoring version constraints entirely will likely confuse users who expect version constraints to be respected and potentially allow conflicting cookbook versions to be used together.
- Environments become less versionable (i.e., it's more difficult to track them as files in git); most likely, you'd upload environment attributes separately from version restrictions, which would in many cases be changing pretty frequently.
- Environments always constrain a cookbook to exactly one version. There's no greater than/less than/pessimistic greater than.
- Compared to tracking an environment as a file in a revision controlled repo, it's trickier to snapshot what version of everything is in use at a given time. Some tooling could help, for example by tagging a git repo with a cookbook's checksum at a particular commit.
- Every cookbook is always frozen, since if it changes it'll have a different checksum. Since cookbooks are individually added to environments, there's much less chance of a dev cookbook being pushed to production.
- Workflow is much simpler. You push a cookbook to the environment where
you want it to go. Compare to editing environment files or JSON or
setting up Ci to compile an environment. Even if you use
-E
and--freeze
withknife cookbook upload
you still have to fiddle the cookbook's version numbers by hand. - More efficient on the server-side: the current implementation requires chef-server to load every version of every cookbook (including dependency information) to compute the correct solution to the various version constraints. This uses lots of database IO and memory, and the resource requirements increase as you have more cookbook versions.
- It's easy to understand what version of a cookbook will be used on a
node. If you add a dependency "B" to cookbook "A", then upload a new
version of "A" without uploading any version of "B", chef-server will
act as if the new version of "A" doesn't exist, because its
dependencies can't be satisfied. Though
knife cookbook upload
guards against this particular case, I've observed similar "why isn't the new version of cookbook X being used" issues that could never be traced to a particular cause. - It's efficient to add a concept of promotion of cookbooks between environments, since you're simply copying one cookbook's checksum to a different environment's constraints.
There are two possibilities for selecting the final set of version constraints in an environment. The first option is an overlay model, where any cookbooks without explicit constraints fall back to using the "_default" environment's constraints. The second is to explicitly version every cookbook in every environment.
In the first model (fallback to default), the version constraints of the "_default" environment are mutable, but only by the Chef server; the Chef server will set them whenever a cookbook is uploaded with no explicit environment.
The set of version constraints is then computed by overlaying the constraints of the node's environment, e.g., "production" on top of the "_default" environment's constraints.
The downside to this model is that the user could introduce a cookbook that should be version constrained into, say, production without setting a constraint. A future update to the non-constrained cookbook could then break production.
In the second model, a cookbook without a constraint in a particular environment would appear not to exist in that environment. This fixes the accidental update issue. On the other hand, it adds friction to user interactions around creating new cookbooks, since a new cookbook has to be added to each environment.
- Add promotion as a first class concept at either the UX level or API level. Single cookbook, multiple cookbook (e.g., single cookbook plus dependencies), and whole environment promotion should be possible.
- Checksums don't sort temporally, so there needs to be automatic metadata to help track/sort which version is when. Timestamp, creator, scm id, etc. could all help here.
- Checksums don't convey semantic meaning, so they should be displayed along with automatic metadata in UIs. Highlighting which versions are assigned to a particular environment would also be helpful.
First of all, I'd like to thank you for opening this discussion up to the public. I appreciate the opportunity to see some of the thoughts behind possible future paths for Chef, and being able to comment on them.
I'm just going to comment on two of the "What You Get" points, followed by some general thoughts.
"Workflow is much simpler."
I disagree. Going this route with changing the cookbook <-> environment interaction, you force a workflow in which "what I have on my workstation" is king. We have a different workflow in which a production-ready cookbook first has to be checked-in to revision control. If you notice while pushing that someone else has already made some changes, they can be merged/discussed/etc. before the upload. Using your proposal, that interaction is now missing. I could have a stale cookbook that just happened to get uploaded to "prod" because I was sloppy in executing the 'knife cookbook upload' from my shell history. Freezing a cookbook and locking that version to an environment by hand is a good thing.
"More efficient on the server-side"
(Please forgive any misunderstandings stemming from my ignorance of the implementation details.) I don't know why the current implementation requires loading every version of every cookbook, or if this is even still the case with Chef 11. Isn't this just metadata? There shouldn't be much I/O or memory requirements to load version and dependency information. The whole cookbook doesn't need to be loaded into memory for this. If the current implementation is inefficient, change the implementation, not the methodology and associated workflow.
General Thoughts
I like having one set of cookbooks and multiple environments. It makes managing the "active set" much easier. knife-boxer looks like a great way to manage a development workflow; I look forward to integrating this into our current workflows. But, it shouldn't be the new way of managing cookbook versions and environments.