File Refactor Announcement

ft-announce.md

Hi Chefs,

We've recently merged a substantial rewrite of Chef's file provider family to master, and it will be released in 11.6.0. As part of these changes, we've standardized the way the different providers create files and update content, and added some cool improvements and features. But first, a word of caution:

Changes to Previously Undefined Behaviors

Prior to this patch, file providers did not have defined behavior for some situations, but instead relied on the behavior of the underlying ruby implementation. We feel that defining and standardizing this behavior will be a huge benefit to you, but there is some risk of breakage, so please read on:

• Chef previously did not define what file permissions it would set if a file resource did not specify them, and was inconsistent between different providers and in some cases differed based on the version of ruby used. All file providers will now create files with default permissions determined by the OS and filesystem default behavior. In general this is governed by your umask setting, but may also be affected by filesystem type or mount options.

  This can be a problem when depending on the default file mode when managing ssh keys or config with cookbook_file or template resources. In previous chef versions, cookbook_file would set the mode of a file to 0600 if not explicitly specified, and template resources would set the mode to 0600 if not specified when running on ruby 1.9.3, but 0644 on ruby 1.9.2 and earlier. In Chef 11.6, these are likely to be created with 0644 permissions (assuming a 022 umask), which can cause SSH operations to error out.

• Chef previously did not have a defined behavior if a file provider encountered something other than a file when attempting to update content. In particular, chef would follow symlinks and overwrite the symlink target's content; other dir entry types (such as devices, named pipes, etc.) would fail in strange ways. Chef will now raise an error instead of overwriting the content of a symlink. If you wish, you can delete whatever's in your way by setting force_unlink true on your resource. Note that there is no longer any built in means for managing the content of a file via a symlink--you must manage the target file instead. UPDATE: In testing, we've discovered that managing a file via a symlink is more common than we thought. For example, Ubuntu 13.04 replaces resolv.conf with a symlink and a cookbook developed for an earlier version of Ubuntu would (mostly) just work with Chef 11.4's behavior. To avoid breaking this use case, we've added an option to manage a symlink's source file and enabled it as the default. This situation will trigger a warning however, and we plan to make this feature disabled by default in Chef 12.

Other Risks

File providers have been significantly refactored, with many methods deprecated. If you've written custom providers that subclass a file provider, we recommend you test them against master (or an upcoming beta when available). We've made every effort to ensure that code using the now-deprecated methods will continue to work, but it's possible that the new code may not set internal state in a way your code expects.

New Features and Enhancements

By rewriting the file providers to comprise a set of reusable components, we're able to deliver a lot of enhancements we think you'll like:

• SELinux support: Chef will restorecon files after modifying them.

• Configurable atomic file updates. Chef lets you choose between atomic (mv-based) or non-atomic (cp-based) file updates. Defaults to atomic. Atomic:

  • will not fail updating important file when out of disk space
  • will not fail updating a running binary
  • may alter file permissions when running as non-root user in some cases
  • files will temporarily have incorrect selinux permissions (until restorecon runs) or windows ACLs (until an ACL restore step runs)

  Non atomic is basically converse of the above.

  Global config: file_atomic_update, per-resource: atomic_update

  NOTE: At the moment, we don't know if this will automatically make community cookbooks work with selinux enabled; it's possible that the debian style file hierarchies used by some community cookbooks conflict with default selinux policy.

• All file providers will now create files such that ACL inheritance works as expected on Windows.

• Remote file can automatically send HTTP conditional GET requests, using ETags and If-Modified-Since. This is enabled by defualt, use use_conditional_get false to disable, or use_etags false or use_last_modified false to disable individual headers.

• Remote file now supports FTP and local files, using "ftp://" and "file://" URIs, respectively.

• Remote file supports custom headers.

Template Helper Methods

Template resources can now define helper methods or modules for use in the template context.

In the template resource, there are three interfaces to extend the template context. The simplest way is to declare a single method inline, like this:

template "/path" do

  # The classic "hello world"
  helper(:hello_world) { "hello world" }

  # You can reference the node object to clean up repeated calls to
  # your cookbook's attributes:
  helper(:app) { node["app"] }

  # Helpers can take arguments.
  helper(:app_conf) { |setting| node["app"][setting] }
end

Your template can then use the methods you've created like this:

Say hello: <%= hello_world %> 

node["app"]["listen_port"] is: <%= app["listen_port"] %>

node["app"]["log_location"] is: <%= app_conf("log_location") %>

If your logic gets more complicated or your dentist told you to avoid syntax sugar, you can define a module inline instead. This may also be handy as an intermediate step between the inline method syntax above and the full-blown module approach below when refactoring. The following code is functionally identical to the first example:

template "/path" do
  helpers do
    # Now you're in the context of a module that will extend your
    # template

    def hello_world
      "hello world"
    end

    def app
      node["app"]
    end

    def app_conf(setting)
      node["app"][setting]
    end
  end
end

Finally, you can simply name a module to extend your template. If you need to reuse extensions in multiple templates, or your template extension code grows too unwieldy for the inline approaches above, you can define a module in a library and use it in your templates like this:

template "/path" do
  helpers(MyHelperModule)
end