ssimeonov/writing-bug-free-chef-recipes.md

## writing-bug-free-chef-recipes.md

      
    Raw
  

              writing-bug-free-chef-recipes.md
            
          
    Writing Bug-Free Chef Recipes

Chef has unique abstraction and processing models. Until you understand them, it is easy to write buggy recipes.
The most important thing to understand is the following: a Chef recipe is not a collection of scripts.
Chef Processing Model

The root of the common confusion about Chef is that the code in Chef recipes is not executed at once but in several passes.
First things first: read about what happens during a Chef run.
The document talks about Chef building a resource list. A Chef resource is something like a file, a directory, a script or a service. Resources are data backed by code (a Ruby class). The code is there to generate the data.
Resources specify the actions that may be performed on them. Most resources have a default action.
Resources do not execute.
Based on the context during a Chef run, a resource, based on its type, name as well as external factors such as the operating system or Chef attributes, is bound to a provider.
The provider is an API (also backed by a Ruby class) that exposes actions. The provider for a service resource may provide actions such as load, start, restart and stop. The implementations for Windows and *NIX are likely to be completely separate. If the action of a resource is one of the actions exposed by its provider, Chef will run the provider method corresponding to the action passing the resource as an argument. Otherwise, Chef will generate an error.
For example, this recipe snippet
file "/tmp/result" do
  action :delete
end
means the following for a Chef run (in a somewhat simplified manner):


Look at the run list, locate cookbooks, identify dependencies and recurse.


Load the libraries folders of all cookbooks on the run list in lexicographic order, which is not the execution order specified by the run list.


Load the recipes in the order of the run list. In the example, at this point Ruby will attempt to execute the method file with arguments ["/tmp/result"] and the code block (a.k.a, callable, proc, lambda, anonymous function) proc { action(:delete) } (or function() { action('delete') } if you want a JavaScript equivalent). You may be wondering which object is the target of the file method? The top-level recipe code is evaluated in the context of a Chef::Recipe object, which uses method_missing, a catch-all for undefined method handling in Ruby.


Ruby's attempt to execute file("/tmp/result", &proc { action(:delete) }) will run current_recipe.method_missing(:file, "/tmp/result", &proc { action(:delete) }). In the body of this method, Chef will use a factory pattern to create a resource object based on the name :file. In the example, this will be a Chef::Resource::File object.


The file resource will be added to the resource list gathered by loading all the recipes on a run list. All resources have IDs. The resource ID is the argument to the resource definition function. You can have two chunks of code in two different cookbooks/recipes. If they define the same type of resource with the same ID, weird things can happen as Chef maintains two lists of resources: by creation order, where duplicates can exist, and by name, where the last one wins.


The code block for file will then run in the context of the newly-created resource object, i.e., the equivalent of just_created_file_resource.action(:delete). This does not delete the file.  All this will do will be to assign the value :delete to the action property of the file resource. Chef uses def foo(value=nil) ... end instead of the more typical def foo ... end and def foo=(value) ... end as a property getter/setter pattern. In Chef, when no value is passed, foo will act as a getter. When a value is passed, it acts as a setter.


Chef will identify a provider type for this resource and will use a factory pattern to create a Chef::Provider::File object.


Eventually, when the time comes to execute the run list, Chef executes provider.delete(resource).


Node convergence & idempotence

It should be clear from the previous section that different parts of the code in a recipe will be run at different times in a Chef run and in different contexts. An added complication is that Chef recipes may be run multiple times on the same node (and across different run lists, environments, etc.).
Chef thinks of building up a node as a process of convergence: the transformation of bare metal to a fully configured system. A key goal in writing simple and bug-free recipes in that context is idempotence. If a recipe is idempotent you can run it one or 1,000 times with the same result.
The ideal way to think of a step in a Chef recipe is that it takes a node from a clearly defined state X (the file /tmp/result is present) to state Y (the file /tmp/result is no longer present). Deleting a file is an example of an idempotent operation. Appending to a file is an example of an operation that is not idempotent: the file size will keep growing. Non-idempotent operations in recipes require special handling to turn them into idempotent steps, typically taking the form of:


Using conditional processing blocks such as only_if and not_if, or,


Using action: none and notifies as the example in the above link shows.


Use resources not scripts

The basic rule of Chef recipes is: try to solve a problem with a resource/provider pairs and not with custom scripting.
Bad:
bash "swoop tmp dir" do
	cwd "/media/ephemeral1"
	code <<-EOH
        mkdir tmp
		chown ec2-user tmp
	EOH
    not_if { ::File.exists?('/media/ephemeral1/tmp') }
end
Good:
directory "/media/ephemeral1/tmp" do
	owner 'ec2-user'
end
This is better for a number of reasons:

It is shorter, simpler and more readable
It communicates intent better
There is a smaller chance of bugs
Many behaviors, e.g., not_if checks come for free, which helps with idempotency and speeds up Chef runs.

Read about the built-in Chef resources & providers.
Lightweight Resource Providers

Resources and providers are the main unit of abstraction in Chef. There is a resource class hierarchy and a provider class hierarchy. The common resources & providers are written in Ruby code as standard classes.
Lightweight Resource Providers (LWRPs) is Chef's attempt to make writing resources and providers easier for people unfamiliar with Ruby. They are basically Ruby files defining classes but without the class XYZ < BaseClass ... end wrapper and with a simple domain-specific language (DSL) for defining actions, attributes, etc.
Writing a custom LWRP is easy.
Common Recipe Bugs

Putting this all together, here are some common patterns for introducing bugs in Chef recipes.
Forgetting when code runs

A natural way to think about conditional execution in normal applications is to wrap a section of code in an if ... end statement as in:
if File.exists?('/var/lib/mongo')
	execute "config replicaSet" do
		command 'mongo --eval "rs.initiate(/* stuff */);"'
	end
end
The problem here is that the if statement will run during the building of the resource list and not during the execution of the recipe. This can lead to subtle Heizenbugs: in this case, an initial run of a recipe from bare metal, when MongoDB is not installed, will skip the Execute resource altogether. However, subsequent recipe runs on the node which now has MongoDB installed will add the Execute resource and run its provider's execute action.
The correct way to add conditional processing is inside the resource's code block:
execute "config replicaSet" do
	command 'mongo --eval "rs.initiate(/* stuff */);"'
	only_if { File.exists?('/var/lib/mongo') }
end