One thing I do not want to do here is reinvent the wheel. The use of resources are well documented here: https://docs.chef.io/resource.html. The concept of idempotence seems to be well explained out there. I encourage you to look that up. It is core to how Chef is supposed to work. Please do not interpret my brevity here to indicate these are not important subjects. You should spend a great deal of time studying resources and how to make them idempotent. I just find the documentation and explanations to be complete and plentiful. I have very little to add.
At this point, I am going to talk about the different types of cookbooks. In one sense, a cookbook is a cookbook is a cookbook. They are all created exactly the same way. However, they fulfill different roles. This is my word. Chef does not make some of the distinctions that I do.
The cookbooks I have presented so far are for building test kitchen instances. Another type of cookbook contain code and resources for configuring clients. I have heard these called library cookbooks. One term used by Chef is 'wrapper' cookbooks. These are cookbooks that call other (library) cookbooks. The behavior of the wrapped cookbook is governed by setting attributes. Another way to think about this is the wrapped cookbook is a bunch of functions and the attributes are parameters for those functions. I like the term library cookbooks for the wrapped cookbooks.
In my opinion, Test Kitchen cookbooks, the ones that build VMs, ought to be treated as wrapper cookbooks. That is, there should be very little code in a cookbook that builds a VM and a library cookbook should never build a VM. One big advantage of working this way is that library cookbooks can be moved into repositories. This makes them readily available to multiple ChefDK workstations.
However, I have found little value in storing the Test Kitchen cookbooks in repositories. For one thing they do not seem to transfer to other ChefDK workstations very well. I often cannot get a cloned Test Kitchen cookbook to create or converge. As a result, I have tended to put a lot of documentation for the configuration of the the wrapper (Test Kitchen) cookbook into the wrapped (Library) cookbook as I copy it up to a repository such as GitHub.
As you will see, the attributes I referred to are put into the Test Kitchen cookbook in order to customize the behavior of the Library cookbook. However, it works best if there is someplace in the library cookbook that contains documentation for the specific use of these attributes. This should become clearer as we work through this tutorial.
- When you move into production, the library cookbooks will get uploaded to your Chef Server and Test-Kitchen wrapper cookbooks get left behind.
- The key to making this work is attributes. Instead of setting attributes in a file within the Test-Kitchen cookbook, these are set on the node, role or environment on the Chef server.
- The rule of thumb here is if you can imagine using the same cookbook to install different roles, then the variance between those roles can be configured with attributes. If your file servers should have different software packages installed based on this role, the list of packages can be stored in an attribute.
- The problems I have had with transferring Test-Kitchen cookbooks between ChefDK workstations has been tracked to variations in drive letters and the inability to put a wrapper cookbook on a different drive than the library cookbook.
In keeping with the desire to do something productive and fun up front, I am going to show you how to install Chocolatey on your Windows Test Kitchen instance and then install a package. We are going to do this using the Chocolatey cookbook available from the Chef Supermarket.
To begin with, we put this functionality directly in the test kitchen cookbook. But very quickly we'll move this to a library cookbook that consumes attributes to control its behavior.
Start by opening the metadata.rb file in the test kitchen cookbook (we are are back to working in c:\chef\azurewindows) for your windows machine and append the following:
depends 'chocolatey'
Then in the recipes\default.rb
include_recipe 'chocolatey::default'
Now run kitchen converge and Chocolatey will be installed. It is just that easy!!
Finally, back in the recipes\default.rb
chocolatey_package 'visualstudiocode'
...and converge again. VS Code will get installed. Note that Chocolatey itself did not get installed on this second convergence. This is an example of idempotence.
There is a fundamental difference between the directory resource we have used before and the chocolatey package we are using now. Directory is a platform resource. It is 'baked in' to Chef. The chocolatey package resource must be downloaded from the supermarket. (There is something called 'berks' that does the downloading a tracking of the supermarket cookbook. We will see more of that later.) Take a look at https://supermarket.chef.io/cookbooks/chocolatey to see more information. While exploring this page also take a moment to click on the View Source button. This will take you to GitHub where this cookbook is maintained. (more on this later) The point I want to make is that once again we included software from a repository with very little effort... just two lines! Then with just one more line, we used the code from one repository to go get software from yet another!
At this point, building a library cookbook just to install Chocolatey and VS Code would be rather silly, but most of the time, there will be multiple packages to install; perhaps a different set for different roles. Rather than remember the package names and options for each package the library itself becomes a great place to make notes. Perhaps more importantly, this library will be kept simple in order to clearly illustrate a number of concepts.
One of the big 'ah ha' moments in learning Chef came when I fully grokked the concept of a library cookbook being a local repository from its inception so that it can be migrated into a public or private GitHub (or similar) repository and then a public or private Chef Supermarket. This means that your brilliant library cookbooks can be shared at multiple levels with at least your organization but also with the entire Internet if you want. Conversely there are Cookbooks available to you in these repositories that were not authored by the folks at Chef and they can be included in your cookbooks as easily as one of Chef's 'baked in' resources. A lot this will become clearer as we walk through the process of building the install-windows-packages cookbook locally and push it up to the Internet.
Note: You will have to know about Git to for this next section!
- Navigate or build an appropriate location on your workstation's file system for your local library cookbooks
chef generate cookbook install-windows-packages- This is exactly the same process as used to create a test kitchen cookbook. It is built as a local Git repository.
- Append to the metadata.rb
depends 'chocolatey' - While in the metadata file, make note of the
version '0.1.0'line. You should be incrementing this when there are changes that could be incompatible with older wrapper cookbooks. - As you make changes to this cookbook, you should be committing the changes to local source control as appropriate to your work style.
- The main functionality for this cookbook will be contained in the default.rb. Here is an example of what that might look like
include_recipe 'chocolatey::default'
if node['Install_Packages']['VSCode'].to_s == 'y'
chocolatey_package 'visualstudiocode'
end
if node['Install_Packages']['Putty'].to_s == 'y'
chocolatey_package 'Putty'
end
if node['Install_Packages']['slack'].to_s == 'y'
chocolatey_package 'slack'
end
# git will not be available to a logged on user (logout/logon to use it)
if node['Install_Packages']['Git'].to_s == 'y'
chocolatey_package 'git' do
options '--params /GitAndUnixToolsOnPath'
end
end
# wants a reboot
if node['Install_Packages']['WinAzPowerShell'].to_s == 'y'
chocolatey_package 'windowsazurepowershell' do
action :install
notifies :reboot_now, 'reboot[Restart Computer]', :immediately
end
end
reboot 'Restart Computer' do
action :nothing
end
As stated above, I am not going to explain the resource blocks and the Ruby code. I found this part of using Chef to be relatively well documented. It is really the use of attributes in wrapper/library cookbooks that I am explaining here and how to design your cookbooks so that they might be as easily consumed as the Chocolatey cookbook itself.
In this example, node['Install_Packages']['VSCode'] is an attribute. Simply put, when this is set to 'y' in the wrapper (test kitchen) cookbook, Visual Studio Code will get installed.
There are a couple of conventions to Git regarding documentation that can and should be utilized when providing instructions. Already included in your cookbook is a README.md file in Markdown format. By placing instructions in this file of the library cookbook you no longer have to remember the syntax details of installing something like Git for instance. In this case, you can include notes about the required attributes. So a portion of your library cookbook README might look like:
Include each of the following lines in an attributes file in order to
install the corresponding Chocolatey package.
default['Install_Packages']['Git'] = 'y'
default['Install_Packages']['VSCode'] = 'y'
default['Install_Packages']['Putty'] = 'y'
default['Install_Packages']['slack'] = 'y'
default['Install_Packages']['WinAzPowerShell'] = 'y'
Note: WinAzPowerShell will invoke a reboot!
Going back to the test kitchen cookbook there are a couple of changes necessary to get it to 'consume' the library cookbook.
- In metadata, change the dependency to
depends 'install-windows-packages' - In the default recipe, change the include to
include_recipe 'install-windows-packages' - delete the chocolatey_package line from the default recipe
Next we need to tell the kitchen cookbook where to find the library cookbook. For this we will turn to the Berksfile. The Berkshelf process is responsible for tracking dependencies and retrieving dependent cookbooks. When we complete this process the test kitchen cookbook will be dependent on the install-windows-packages which will be dependent on the chocolatey cookbook.
For now, add the following line to your Berksfile. Note the forward slashes here, not backslashes. Just another reminder that Chef was designed for Linux!
cookbook "install-windows-packages", path: "[drive-letter]:/[path]/[to]/[cookbooks]/install-windows-packages"
Finally, we add the attributes 'scaffolding' and set the attributes for the packages we want to install. Descend into the top level of your cookbook and enter
chef generate attribute default
Now edit the newly created attributes\default.rb file
default['Install_Packages']['Git'] = 'y'
default['Install_Packages']['VSCode'] = 'y'
default['Install_Packages']['Putty'] = 'y'
or whatever combination of attributes you desire.
Run kitchen converge on the test kitchen cookbook. This will call the recipe in the library cookbook and the packages you call out with attributes will get installed. Congratulations, you have now made your kitchen cookbook into a wrapper cookbook to the 'install-windows-packages' library cookbook!
To wrap up this section, it is possible to put multiple recipes in a single cookbook. The include_recipe command that gets placed in the default recipe of the kitchen cookbook implicitly calls the default recipe of the library cookbook. The following two lines are equivalent:
include_recipe 'install-windows-packages'
include_recipe 'install-windows-packages::default'
We could build a much more extensive cookbook, perhaps 'windows-installation-recipes' and copy the default recipe from the Chocolatey cookbook and make it one of many recipes in the 'windows-installation-recipes'. Then do the same thing for Ubuntu server etc.
The process I have described is great way to start the development of a library cookbook. Its limitation is that it only exists on your local ChefDK workstation. This might be fine if you are working alone. However, by making cookbooks into local Git repositories by default, all manner of collaborative opportunities become available in a manner well known to a lot of developers.
At this point, I am going to discuss the utilization of a public GitHub repository for collaborating and even including in your kitchen cookbooks directly. Somewhere along the way in your development of a library cookbook, there will be an initial version that appears to work. This would be a good opportunity to copy/sync/push this version up to a public or private repository. If you have not already, create a GitHub repository for this local repository, set the location of this in the local Git repository and push (sync) the contents up to it. If none of this makes any sense, then please take the time to learn Git!
Now, this library cookbook can be consumed by any ChefDK workstation that has access to the Internet! This means that you can work with this cookbook from multiple computers or other people can use your cookbook and even suggest changes/fixes/improvements through built in GitHub functions. All it takes is an adjustment to the Berksfile:
cookbook "Install_Packages", "~> git: "https://github.com/[user]/[repo_name].git", ref: "[commit id]"
Notice that you can pin your kitchen cookbook to a particular commit (version) of the library cookbook. This is ideal when collaboratively working with a cookbook under development.
The point of Berks seems to be to take care of the details and complexity cookbook dependencies. So one should not have to study it in detail. As long as that seems to hold true, this appendix can be ignored. But when things go south, here is a walk through that might help.
Enter berks help to see the commands.
Starting in a newly created cookbook enter berks list. The result will be:
Lockfile not found! Run `berks install` to create the lockfile.
Enter berks install
Resolving cookbook dependencies...
Fetching '[cookbook name]' from source at .
Fetching cookbook index from https://supe
Using [cookbook name] (0.1.0) from source at .
Notice that a Berksfile.lock file was created. Now berks list just includes the local cookbook.
Now add a dependency in metadata such as depends chocolatey and run berks list again
The lockfile is out of sync! Run `berks install` to sync the lockfile.
berks install again:
Resolving cookbook dependencies...
Fetching 'ServerX6' from source at .
Fetching cookbook index from https://supermarket.chef.io...
Using ServerX6 (0.1.0) from source at .
Using ohai (5.2.0)
Using chocolatey (1.2.1)
Using windows (3.4.0)
Converging a cookbook after a change to the metadata implicitly runs berks install.
berks list and the contents of Berksfile.lock will confirm these new dependencies.
Cookbooks are downloaded and stored in the .Berkshelf directory. In Test Kitchen for windows this is located in the windows profile for the user running Kitchen.