Policies and Cookbooks and Groups

policies.md

The policy resource

You can create Chef machine policies (essentially pinned resources) using the chef_policy resource. When it is done, the policy, and any referenced cookbooks will be on the Chef server.

Specifying via attributes

You can define the policy inline in the file. The most basic version:

chef_policy 'name' do
  recipe 'base'
  recipe 'blah::x'

  cookbook 'base', '>= 1.1.0'
end

chef_policy 'name' do
  run_list 'recipe[base]', 'recipe[blah::x]'
  cookbook 'base', '>= 1.1.0'
end

You can pull the policy from a Policyfile, as well:

chef_policy 'web' do
  based_on_policyfile 'web'
end

# Shorthand
chef_policy 'web.rb'

Default source

The default source for cookbooks is the https://supermarket.getchef.com community site. You can override the default source for cookbooks for a policy in config:

# .chef/knife.rb
default_cookbook_source :community

You can put it in the recipe:

with_cookbook_source :community

chef_policy 'name' do
  recipe 'base'
end

chef_policy 'another' do
  recipe 'base'
end

You can even put it on each resource:

chef_policy 'a' do
  cookbook_source :community
  recipe 'base'
end

chef_policy 'a' do
  cookbook_source :chef_server
  recipe 'base'
end

Inheriting policies

You can inherit one policy from another using based_on. When this happens, the new policy will have the exact versions of any cookbooks specified in the base policy, and will include any run_list as well.

chef_policy 'base' do
  recipe 'base'
  cookbook 'base', '> 1.0.0'
end

chef_policy 'web' do
  based_on 'base'
  recipe 'apache2::install'
end
# base version is same as whatever is in the base policy
# Runlist: recipe[base], recipe[apache2::install]

You can reorder parts of the original run_list by writing down the order of the things you care about:

chef_policy 'base' do
  recipe 'a'
  recipe 'b'
end

chef_policy 'web' do
  based_on 'base'
  recipe 'c'
  recipe 'b'
  recipe 'a'
end
# Results in c, a, b

chef_policy 'web' do
  run_list ['recipe[c]', 'recipe[a]']
end
# Results in c, a, b

Lockfiles

Using lockfiles to create policies

You can manage lockfiles out of band, or explicitly, and create a policy from a lockfile:

chef_policy 'web.lock.json'

chef_policy 'web' do
  based_on_lockfile 'otherbase.lock'
end

Creating lockfiles

This has the same idempotency rules as chef_policy, and all the same attributes are valid.

chef_policy_lockfile 'web.lock.json' do
  default_source
  recipe
  cookbook
  based_on
  based_on_lockfile
  based_on_policyfile
end

Using policy in a machine or node

To use a policy in a machine or a node, you just put the attribute there:

machine 'blah' do
  policy 'web'
end

This attribute is mutually exclusive with node run_lists and attributes--it is an error to specify both on the node or machine.

Idempotency

The resource has to make decisions on when and what to update. Depending on how often you want your machines to change, there are several strategies for doing so.

Aggressive updates

With aggressive updates, we recalculate the policy lock every time and update the server anytime it has changed. This way, it picks up the latest version of everything, every time (which may or may not have changed). This is usually what you want; if you need to make sure things are pinned the same way from group to group, you can use based_on or based_on_lockfile to ensure version pins don't move.

Conservative update

The conservative update strategy tries not to update things unless it has to. Thus, if a cookbook already has a version in the server policy, we will update it only if it cannot satisfy local constraints.

If there are cookbooks in your local chef repository, it treats those as = constraints.

For example, let's say both your security and apt_get cookbooks are behind. Both are at version 1.5 on your server, and 2.2 has been released. This will trigger an update:

chef_policy 'base' do
  recipe 'security::hardening'
  recipe 'apt_get'
  cookbook 'security', '> 1.0'
  updates :conservative
end
# security and apt_get are now both 2.2

And this will not:

chef_policy 'base' do
  recipe 'security::hardening'
  recipe 'apt_get'
  cookbook 'security', '> 2.0'
  updates :conservative
end
# security and apt_get are both still at 2.2

Ultra-conservative updates

Ultra-conservative updates are like the conservative update strategy, except even after the decision has been made to update the policy, it still tries not to update versions unless they cannot satisfy constraints. based_on_* is probably enough to handle this sort of scenario, and this strategy is probably very hard to implement, so we won't have it anytime soon.

For example, let's say both your security and apt_get cookbooks are behind. Both are at version 1.5 on your server, and 2.2 has been released. This will trigger an update:

chef_policy 'base' do
  recipe 'security::hardening'
  recipe 'apt_get'
  cookbook 'security', '> 1.0'
  updates :ultra_conservative
end
# security is 2.2, but apt_get is still 1.5

And this will not:

chef_policy 'base' do
  recipe 'security::hardening'
  recipe 'apt_get'
  cookbook 'security', '> 2.0'
  updates :ultra_conservative
end
# security and apt_get are both still at 2.2

Purging old cookbooks

By default, the chef_policy will not "garbage collect" any old cookbooks. To force a garbage collection, you can use this resource:

chef_cookbooks_garbage_collector 'blah'

It will look through all policies on the server, and delete any cookbook versions that are no longer used.

policy_groups.md

Policy groups

Policy groups are a new construct that essentially create a namespace for data, nodes, clients and policies. The primary purpose of this is to allow distinct data, nodes, clients and policies for distinct policy groups.

Policy groups are similar to environments, except that they actually provide a walled-off namespace so that you can have things with the same name in each of your dev, test and prod policy groups.

Using policy groups: metal side

There are several ways ot set the policy group. You can set policy_group in your knife.rb or client.rb file:

# In .chef/knife.rb
policy_group 'prod'

You can use with_policy_group to affect cheffish and chef-metal:

with_policy_group 'prod'

# Machine
machine 'db'
chef_data_bag_item 'secrets/db'

with_policy_group 'test'

machine 'db'
chef_data_bag_item 'secrets/db'

And you can set them directly on each machine:

machine 'db' do
  policy_group 'prod'
end
chef_data_bag_item 'secrets/db' do
  policy_group 'prod'
end

# These will be a different machine and data bag
machine 'db' do
  policy_group 'test'
end
chef_data_bag_item 'secrets/db' do
  policy_group 'test'
end

What they affect

Policy groups affect data bags, nodes, clients, policies, and roles. It does not affect environments, cookbooks, or users.

Additionally, there are still global data bags should you need them.

Using policy groups: client side

Chef core APIs will all be adjusted to work with policy_group and understand the prefixes. Your node will have the policy group marked on it, so this will be slurped in at the beginning of the run. (How it is marked will be different in Chef 11 and Chef 12.)

ChefDataBagItem.new('secrets', 'db')
search(:node, ...) # Not sure how effectively we can do this.  But we can try in Chef 11 anyway.

Retrieving a global item means temporarily setting Chef::Config.policy_group to nil:

Chef::Config.policy_group = nil
begin
  ChefDataBagItem.new('users', 'blah')
rescue
end

ze_cookbook.md

Cookbook resources

There are still going to be people not using policies. If you want to get a cookbook and all its dependencies, you can get it to the server thus:

chef_cookbook 'cookbook'

If the cookbook is in your local chef repository, it will upload; otherwise, it will upload from the community server.

You can also specify the actual path:

chef_cookbook 'cookbook' do
  source_path 'path/to/cookbook'
end

# Shorthand
chef_cookbook '/path/to/cookbook'

From community

If you want a cookbook from the Supermarket, you can get it to the server like this:

chef_cookbook 'apache2' do
  source :community
end

No dependencies

Sometimes you just want the one cookbook regardless of deps. To do that, run this:

chef_cookbook 'apache2' do
  dependencies false
end

Inline cookbooks

You can also directly specify the content inline:

chef_cookbook 'cookbook' do
  metadata do
    version '1.0.0'
  end
  recipe 'name', 'content'
end

This feature lets you write recipes in your recipe, so that you can define a
metal world without leaving your file.  Good for learning, not so good for
production.