README.md

Blubber

Blubber is a tool for creating a Docker container description file from a manifest configuration file describing one or more variants and an optional policy validation ruleset.

Usage

$ blubber blubber.example.yaml development [file:///path/to/policy-file.yaml] > Dockerfile

Manifest File

The manifest file declares the overall intention for a blubber distribution. It comprises a base configuration with values that can be replaced or enhanced by one or more variants' configurations.

Variants

Variants provide a means to define more specific configurations for specialized contexts (i.e., testing, development, production). They can inherit and extend properties from the base configuration or other variants.

Sample Configuration File (YAML)

---
version: v2
base: debian:jessie
apt:
  packages: [libjpeg, libyaml]
python:
  version: python2.7
runs:
  environment:
    FOO: bar
    BAR: baz

variants:
  build:
    apt:
      packages: [libjpeg-dev, libyaml-dev]
    node:
      requirements: [package.json, package-lock.json]
    python:
      requirements: [requirements.txt]
    builder: [make, -f, Makefile]

  development:
    includes: [build]
    sharedvolume: true

The base configuration specifies a Python 2.7 environment on top of Debian Jessie, with libjpeg and libyaml installed, and FOO and BAR set in the application's environment

There are two variants: build and development. The build variant specifies that in addition, further apt packages, node packages, and the manifest requirements.txt file should be installed. The process to drive that will be an included Makefile.

The development variant includes all of the build variant, with the additional configuration that it should be done on a shared volume.

Policy File

A policy file defines constraints that can be imposed on the process that creates the Docker configuration. It consists of a number of enforcements, which map a configuration path to a rule to be applied to validate it. In addition to the validator.v9 rules, Blubber provides additional rules specific to its configurations:

• abspath: true if the value is a valid absolute, non-root path
• baseimage: true if the value is a valid base image reference
• debianpackage: true if the package name conforms to Debian's package naming convention
• envvars: true if the environment variables' names conform to proper naming convention
• isfalse: true if the value is Boolean false
• istrue: true if the value is Boolean true
• variantref: true if the value is a valid variant reference
• variants: true if the variant names conform to proper naming convention

Sample Policy File (YAML)

enforcements:
  - path: variants.production.base
    rule: oneof=debian:jessie debian:jessie-slim
  - path: variants.production.runs.insecurely
    rule: isfalse

This policy file constrains the base image for a production variant to be one of jessie or jessie-slim. Any attempt to use an alternate image would result in an invalid operation. Furthermore, the production variant cannot be run insecurely.

Areas for Improvement

[First, some caveats: While I took the opportunity to embark on a crash-course in Go, I'm unfamiliar with the broader (and much more deep and pervasive) idiomatic approaches of the language. My notes here won't reflect anything that is counter to typical linguistic conventions, and will focus on potential room for systemic improvements.]

Blubber, as it was presented to me, sits in a bit of a vacuum. I'm not entirely certain how it's used in the production pipeline so I'm unclear on where the configuration files typically reside. They're read directly by opening the path to the file on a mounted filesystem. It seems they could benefit from existing in a known centralized repo. Perhaps they already do? If that were the case, they could be referenced by URL or even by a unique short name identifier. Policy files are already anticipated to be URIs. This allows them to be updated remotely and constraints updated without propagation. A similar approach could be taken with configuraiton files.

Similarly, the binary expects to make use of typical shell stdout for the resulting Dockerfile. This might be Go convention as well, or it could be that the consuming machinery surrounding Blubber already anticipates and makes use of stdout.

Process notes

I did checkout Blubber directly from Phabricator, and I saw there was a README.md, but renamed it locally and studiously avoided looking. I also used the opportunity to see that the export you provided was the most recent version of the code. This improved the confidence I had in documenting the system because I knew it would reflect the current state of the project.