Skip to content

Instantly share code, notes, and snippets.

@timfoster

timfoster/lullaby-phase-4-flag-day.md Secret

Last active May 15, 2019 22:00
Show Gist options
  • Save timfoster/fb31254fd726af217397da40b622c637 to your computer and use it in GitHub Desktop.
Save timfoster/fb31254fd726af217397da40b622c637 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
lullaby-phase-4-flag-day.md

Hi there,

If you build any of the following git repositories, you should read this flag day mail:

  • smartos-live.git
  • sdc-headnode.git
  • firmware-tools.git
  • mountain-gorilla.git

We'll describe the changes to each component in turn. There are no changes needed to any development zones used for building these repositories.

Broadly, the changes alter the build and release engineering process within Joyent, removing the requirement to use "Mountain Gorilla" to perform official builds. Read past the 'tl;dr' section below if you want the details.

tl;dr

This is a very quick summary of the developer-impact of these changes.

For all components, we're replacing the old $TRY_BRANCH method of selecting which branch to build with a means of specifying exactly which branch(es) should be used.

  • smartos-live

    Stop using configure.smartos (and sample.configure.smartos) and instead use configure-projects. E.g.:

    $ cp default.configure-projects configure-projects
(modify configure-projects to select the projects branches to build)
$ make world && make live

    The previously slightly different SmartOS and Triton platform build flavors are now combined into the same image. Additional Makefile targets are available to build and upload specific sets of artifacts for those products.

  • sdc-headnode

    You may still use build.spec.local, but can now also use a simpler configure-branches file to set the branches of components to be included in the headnode build.

  • mountain-gorilla

    The launch-build script now takes a -F option to select which artifacts should be produced for the platform build.

smartos-live.git

A description of the changes

The main change to the SmartOS build is related to how we configure the projects framework, which is used to host source trees needed when building a SmartOS or Triton platform image.

We're dropping configure.smartos, previously used to configure the branches and git URLs for the projects framework and replacing it with configure-projects and the optional configure-build files.

We supply default files default.configure-projects and default.configure-build at the top-level of the repository.

The configure-projects file is a colon-delimited file consisting of 3 fields. Here's an example:

   illumos-extra: master: origin
   illumos: master: origin
   local/kvm-cmd: master: origin
   local/kvm: master: origin
   local/mdata-client: master: origin
   local/ur-agent: master: origin

The first and second fields are the local project directory and branch at which we should include them. The third field should be a git URL indicating where we should clone the repository from. The special keywords origin and cr indicate we should pull from the main joyent github.com or joyent gerrit repository URLs.

As another example, here's how we configure the build to pull from local illumos and illumos-extra repositories rather than github:

   illumos-extra: master: /home/timf/projects/illumos-extra.git
   illumos: master: /home/timf/projects/illumos-joyent.git
   local/kvm-cmd: master: origin
   local/kvm: master: origin
   local/mdata-client: master: origin
   local/ur-agent: master: origin

If you have been using Joyent's Jenkins CI system, the platform and platform-debug jobs have been modified to take a CONFIGURE_PROJECTS parameter to set the contents of the configure-projects file.

By setting different values for the branch and git URL fields, you can specify exactly which git branches should be built for the different projects repositories. (previously Mountain Gorilla used a $TRY_BRANCH mechanism which would attempt to checkout repositories at given branches, before falling back to the current branch - this change removes that functionality)

We're now following the same pattern as the headnode build when naming platform build artifacts, by including any additional branch names as necessary - see the next section sdc-headnode for more background.

Similar to other projects across Manta/Triton, we're adding eng.git as a submodule to smartos-live.git, which will be automatically cloned to ./deps/eng when you run make.

smartos-live now contains all of the SmartOS-specific release tooling, generating index.html fragments, and forging the correct snaplinks in Manta if this is a release build, which form the basis of the SmartOS release URLs.

We're also unifying the smartos and platform Jenkins jobs builds, deleting the smartos job and instead always choosing to bundle ur as part of the release. A new Jenkins parameter PLATFORM_BUILD_FLAVOR can be set to declare which artifacts should be generated - Triton platform images, SmartOS images, or a combination of those.

The default is to generate only Triton platform images.

All of the logic contained in mountain-gorilla.git to drive the various platform build flavors and to create the Triton platform image manifests has been included in smartos-live.git.

This resulted in the addition of targets to build the appropriate artifacts and upload those to Manta (by default) See the 'Build Targets for Release Engineering' section of README.md for more information.

  • triton-release
  • smartos-only-release
  • triton-and-smartos-release

A shell wrapper suitable for invoking via Jenkins is at ./tools/build_jenkins

Despite these changes, you can continue using the same ./configure ; gmake world && gmake live process that existed before this change.

You can now also use the print-* target to emit Makefile macro values, for example:

$ make print-BOOT_MANIFESTS
BOOT_MANIFESTS=/space/timf/smartos-live/boot.manifest.d/illumos.manifest

Full documentation for the build changes has been included in README.md at the top-level of the smartos-live.git repository.

Converting existing workspaces

As we discussed, the role of configure.smartos (a shell script) is being taken on by configure-projects (a colon delimited text file).

You should manually convert any configuration between those two files. If you previously had custom shell logic in configure.smartos, that can now be set in configure-build, possibly overriding the defaults in default.configure-build. It's unlikely you'll need to change these.

You should not need to clobber workspaces after pulling these changes, but should re-run ./configure having updated your config files.

sdc-headnode.git

Described in TOOLS-2238, the standard branch naming scheme for build artifacts that are composed from multiple repositories on (possibly) different branches was too simplistic.

In addition, the fallback mechanism of $TRY_BRANCH isn't sufficient since when looking at build artifacts, it's not obvious which branches were used for the build (e.g. did a TRY_BRANCH artifact not actually exist for a given component, and we ended up using a 'master' build instead?)

Apart from adding eng.git as a submodule of sdc-headnode, we're modifying the headnode build so that it's easier for developers to construct builds using components from different branches. The headnode build already had a means of doing this, with 'build.spec.local' files, but to make it easier for developers, we're following a pattern already used by smartos-live, by introducing a shortcut in the form of a 'configure-branches' file, which, along with build.spec.local, is used to generate a build.spec.merged file at build-time should one not already exist. We're dropping the other merge mechanisms used in shell scripts in favour of simply referencing build.spec.merged instead.

If the correct bits are not found for a given component, the build intentionally fails fast, rather than driving on.

Adjustments to the Jenkins jobs for the headnode builds were made. See:

If builds are done that reference components from more than one differently named branch, the branch value used by the build is:

    [branch of sdc-headnode]-[uniquified list of additional component branch names]-[timestamp]-[githash]

If we only have a single branch, then we use the current branch naming scheme:

    [branch of sdc-headnode]-[timestamp]-[githash]

You can see some example builds at this Manta path:

    /Joyent_Dev/stor/timf-builds/headnode
    /Joyent_Dev/stor/timf-builds/headnode-debug
    /Joyent_Dev/stor/timf-builds/headnode-joyent

By default, build.spec is setup to pull artifacts from the master branch for inclusion in the headnode build. To build a release version of the headnode image, pass a CONFIGURE_BRANCHES parameter (which gets written to configure-branches in the workspace like:

    bits-branch: release-20190425

To build a headnode build with specific components use a configure-branches file like:

    bits-branch: release-20190425
    platform: master
    imgapi: grr-TRITON-12345

In the above example, we're using release-20190425 for most components, the master branch for platform (and related platform components, like platimages and platboot which get auto-completed) and the grr-TRITON-12345 branch component for imgapi.

As with other the lullaby conversions, publish and bits-upload Makefile targets allow you to post build artifacts to Manta.

firmware-tools.git

This is a standard conversion of the existing firmware tools repository, adding eng.git as submodule under ./deps/eng in the repository, allowing us to build and publish bits to Manta directly using the 'bits-upload' target rather than needing mountain-gorilla.

mountain-gorilla.git

The changes here are to remove the smartos-specific release scripts from MG, which were moved to smartos-live.git, as well as to change the 'launch-build' script, so that we're passing the correct parameters to the 'platform' build and 'headnode' builds. See the sections below for details of what's changing there.

Bugs fixed

The following bugs are being fixed as part of this final phase of the lullaby-3 project:

  • TOOLS-2229 MG's launch-build script needs to adapt to lullaby-phase4
  • TOOLS-2228 convert headnode build to engbld framework
  • TOOLS-2220 convert firmware-tools to engbld framework
  • TOOLS-2071 headnode build should work with a local manta-style BITS_DIR
  • OS-7453 convert platform build to engbld framework
  • OS-6986 SmartOS release builds should include Ur

Please let me know if you've got any questions or concerns. This is the last major phase of the "Lullaby 3" project that I've been working on since I started at Joyent just over a year ago. Phew!

I'd like to say a special thanks to everyone who's encouraged, supported and advised on this work along the way - your help was very much appreciated :-)

cheers,
		tim

This flag day mail is archived at

https://gist.github.com/timfoster/fb31254fd726af217397da40b622c637

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment