Nodes proposal

gistfile1.md

Node Proposal

The "node" concept in Chef right now suffers from a conflation problem between desired node state, and actual node state. This causes a couple of problems:

• Users configure nodes (run lists, tags, attributes) and it sometimes gets overwritten by chef-client
• When editing and viewing node configuration, the data in some views are heavily confused because it is overrun by ohai attributes

Solution: split node configuration (stuff the user passes as input for the node) from runtime data (stuff detected by ohai and spit out at the end of a run) in the Chef server to avoid the config data getting overwritten when data is output (and vice versa). config and runtime are merged during GET /nodes/NAME so that things work seamlessly with Chef 10 and 11 clients.

/nodes/NAME/configuration endpoint

Add a new endpoint for user-editable node configuration, called /nodes/NODE_NAME/configuration. This endpoint accepts POST, PUT, and GET. POST will create a node and update only configuration data. PUT and GET act only on this configuration data, which looks like:

{ "name": "node_name", "chef_environment": "_default", "run_list": [ "recipe[recipe_name]" ], "normal": { "tags": [ ] } }

/nodes/NAME/runtime endpoint

The /nodes/NAME/runtime endpoint will have the attributes that are generated by a Chef run, and accept POST, PUT and GET. POST will create a node and update only runtime data.

{ "name": "node_name", "default": { }, "override": { }, "automatic": { } }

/nodes/NAME endpoint

The /nodes/NAME endpoint will accept GET, POST, PUT and DELETE and work exactly like it currently does.

chef-client vNext

In vNext, chef-client will GET runtime and configuration data separately, and at the end will PUT either /nodes/NAME/runtime (if configuration data has not changed, the normal case) or PUT /nodes/NAME (if configuration data has changed).

knife vNext

knife upload, download, etc. (essentials) in vNext will put node configuration data in /nodes/NAME.json, and will relegate runtime data to /nodes/runtime/NAME.json. There will be no combined view.

knife node edit, show, and friends will work with configuration data. knife node edit, show, etc. with --runtime will work as it does now (on all data).

knife run_list commands will work with configuration data.

knife tag commands will work with configuration data.

People currently do things like mutate the run_list in a client run, as well. Maybe the client should track if it dirties any of its component data (i.e., attributes in the OO sense)?

Ah, I didn't add that detail, thanks!

Related to the ideas described here:

If configuration is separated from runtime, perhaps the client should GET only the configuration at the start of a run. I think at present, the client throws away the ohai data and regenerates it at the beginning of a run anyhow.

In terms of implementation detail, I wonder if we might consider storing the full configuration + runtime data only in the search index (in a way that the data could be retrieved).

I didn't see your comment seth, that totally makes sense. As for implementation details, I can see it being done either way. Storing the data separately would be faster, I'm quite sure. The benefit of having the data stored together is it wouldn't require an upgrade procedure.

Should default and override attributes be configuration or runtime data?

default and override attributes are runtime data. they're wiped at the start of the run and fully regenerated by the chef run and then POST'd at the end.

normal attributes (including the run_list and tags) are the configuration data.

the chef-client should definitely do a GET of only /nodes/name/configuration at the start. if it 404's it should probably GET /nodes/name for backcompat. that'll avoid pulling all the automatic/default/override attrributes which is fully 98% of the node data and avoid us parsing all of that data into ruby objects and then immediately throwing all that work away.