Draft release notes for Bonsai

release-notes.md

Garden Bonsai (0.13) has been released!

This is a big new release, and we've been hard at work on it for several months. The main pieces are:

• Actions. Backwards-comatible with modules, these are a simpler and more flexible way to describe your dev workflows.
• An improved CLI experience. This includes a reimagined interactive dev command.
• Start/stop code syncing without redeploying. The sync start & sync stop commands give you finer-grained control over code syncing.
• Re-run tests and scripts without rebuilding. The inner loop of development just got even faster.

Below, we dive into these in a more detail.

We'd love to hear your thoughts and feedback! If you encounter any bugs or have ideas for improvements or additional features, please open a GitHub issue for it.

For those of you who already have Garden projects from earlier versions, we recommend taking a look at the Bonsai migration guide in our docs.

Finally, we warmly welcome you to our Discord community, where we also do our development out in the open!

Breaking changes

Here is the list of breaking changes in Bonsai:

• The cert-manager integration has been deprecated.
• garden delete has been renamed to garden cleanup
• garden delete env has been renamed to garden cleanup namespace with an alias of garden cleanup ns
• dotIgnoreFiles has been renamed to dotIgnoreFile and only supports one file.
• modules.* in project configs has been renamed to scan.*.
• The deprecated hot-reload mode has been removed—please use sync instead.
• The deprecated cluster-docker build mode has been removed, use cluster-buildkit or kaniko instead.
• Dropped support for deploying an in-cluster container registry.
• Dropped support for the following providers:
  • google-app-engine
  • google-cloud-functions
  • local-google-cloud-functions
  • maven-container
  • npm-package
  • openfaas

Actions: A simpler, more flexible way to describe your pipelines

Abstractions are all about noticing patterns.

That's where Garden's design came from—we noticed that almost all dev automation and CI pipelines boil down to a sequence of a few basic types of operations:

• Builds
• Deploys
• Tests
• Runs (for all those pre/post steps, e.g. to set up auth, or load DB schemas and data)

All of these have slightly different semantics (deploys, for example, have "liveness" semantics about being running or missing or unhealthy, whereas tests and runs only succeed or fail).

A framework that understands those semantics enables you to be a lot more expressive when defining your pipelines. This means way less boilerplate, and way less time spent debugging. Making life easier for engineers by capturing these commom semantics in a batteries-included platform was why we built Garden to begin with.

Until now, Garden has grouped these operations into modules. The idea was that code and the operations performed on that code (building, deploying, testing and running) belonged under the same umbrella (a module).

Over time, we realized that it was simpler and more flexible to configure each of these operations as its own action (essentially breaking modules up into their constituent operations). Conceptually, this:

module
|-> build
|-> one or more deploys
|-> one or more tests
|-> one or more tasks (now runs)

now just becomes a list of Build, Deploy, Test and Run actions. See the before & after section below for a more detailed example of this.

Actions are very close to how people already model the domain instinctively: What sequence of steps do I need to spin up this environment? Each of those steps now neatly fits into an action.

In the example below: We have a Docker image for a service that's deployed via a Helm chart. We've got unit tests and integration tests for the service, and we'd like to be able to do code syncing on the service.

Mapping this to actions is a lot more straightforward than it was with modules: A Build action for the Docker image (build.vote-image), a Deploy action for the Helm chart (deploy.vote) and Test actions for the unit & integration tests (test.vote-unit and test.vote-integ). These are all you need for a CI pipeline to deploy & test this service—or to deploy a dev environment with code syncing!

The net result: Going forward, it will be a lot easier to lay out your CI pipelines and dev automation with Garden.

That's the beauty of composable abstractions: No matter how complex your stack and your pipelines are, you can see at a glance how they decompose into a graph of actions.

It's a lot easier to build castles out of Lego than out of sand!

Actions are backwards-compatible with modules

Module configs still work in Bonsai—internally, Garden simply converts modules to their constituent actions on the fly. Over time, we recommend that you convert your module configs to action configs, since there are new and exciting capabilities that are only available in actions.

Any action can depend on any action

Dependencies have become a lot more flexible in Bonsai: Any action can depend on any action.

This is very useful e.g. when designing CI pipelines, where you only want to start an expensive Docker build if unit tests pass, and where you may need a run step to prepare things before a build or deploy can take place (e.g. run a script to configure authentication, or dynamically generate a configuration file that's used by a build).

A reimagined garden dev command—now interactive!

The garden dev command has been redesigned from the ground up.

When you run it, it loads your project, and opens an interactive prompt. You can then build, deploy and test, start or stop code syncing and stream service logs without exiting the command.

Since you don't need to load your project every time you want to perform an action, this makes for a very smooth inner-loop experience —especially when used in conjunction with the new sync start/sync stop commands and the new rebuild-free kubernetes-exec tests and runs (more on those below).

You can get the same fast feedback loops when code syncing and running integ/end-to-end/API tests in a production-like environment as you're used to when running and modifying unit tests for a single locally-running service.

All in all, we think the new dev command is a big step towards a truly first-class developer experience for Kubernetes!

Note: If you run garden deploy --dev|sync|forward, a dev command session that starts by running the requested command is launched instead. This means you always get the full-powered interactive experience, no matter which persistent command you use as the entrypoint.

Code syncing: More control and visibility

The new sync start and sync stop commands can be used to stop and start code syncing for one or more deploys.

When run inside the garden dev command, the syncs are stopped when the command terminates. When run outside the dev command, by default they stay running after the command terminates.

To see the status of any code syncs you've started, you can use the garden sync status command.

This will also show you diagnostic messages for any syncs that have been halted, which can happen if you move sync roots around or delete entire directories). The garden sync restart command is a convenient way to restart syncs after you fix any sync issues.

All in all, being able to stop and start syncs without having to redeploy can save a lot of unnecessary rebuilding during development, which will hopefully come in handy for all of us using Garden's code syncing throughout the workday!

Lightning fast tests & runs with kubernetes-exec

Bonsai ships with two types of test and run: kubernetes-pod and kubernetes-exec.

kubernetes-pod is the classic type of test and run, familiar to users of 0.12. When a kubernetes-pod test or run is triggered, a one-off Pod using the appropriate image is deployed, and the test/run's command is executed in the pod (after which is it terminated).

While this is a great fit for most CI use-cases, it isn't ideal for use with code syncing, since re-running a test/task will require rebuilding the image if any changes have been made.

kubernetes-exec is a new type of test and run. These execute a command in an already running pod and stream the logs back. This is perfect for:

• Rapid testing when using code syncing.
  • Re-run test suites without going through a full Docker build every time, and without having to deploy a one-off test pod.
  • This is the perfect companion to code syncing, and gives you the same fast feedback loops when running integration/end-to-end/API tests as you're used to when running unit tests locally.
• Speeding up deployment pipelines that require Run steps.
  • When performing operations like seeding a database with a schema and test data, doing so in a one-off pod tends to be slower than necessary. kubernetes–exec Runs can speed things up by simply executing a command in the running pod.
  • This is also a great fit for quickly executing common commands in a running pod, e.g. to reload a dev database or to generate database migrations.

All in all, kubernetes-exec actions can make some of the most common dev workflows with Kubernetes an order of magnitude faster than previously possible—from minutes to seconds.

In detail: A before & after example of modules converted to actions

Here's a "before & after" example (from the vote-helm project):

############
# 0.12-style
############

# examples/vote-helm/vote-image/garden.yml
kind: Module
description: The voting UI container image
name: vote-image
type: container
tests:
  - name: unit
    args: [npm, run, test:unit]

# examples/vote-helm/vote/garden.yml
kind: Module
description: Helm chart for the voting UI
type: helm
name: vote
devMode:
  sync:
    - target: /app/src
      source: src
      mode: two-way
serviceResource:
  kind: Deployment
  containerModule: vote-image
dependencies:
  - api
values:
  image:
    repository: ${modules.vote-image.outputs.deployment-image-name}
    tag: ${modules.vote-image.version}
  ingress:
    enabled: true
    paths: [/]
    hosts: ["vote.${var.baseHostname}"]
  env:
    - name: HOSTNAME
      value: vote.${var.baseHostname}
tests:
  - name: integ
    args: [npm, run, test:integ]
    timeout: 60
    dependencies:
      - vote

###################
# 0.13/Bonsai-style
###################

# from examples/vote-helm/vote-image/garden.yml
kind: Build
description: The voting UI container image
name: vote-image
type: container

---

# Tests now have their own config.
kind: Test
name: vote-unit
type: container
build: vote-image
spec:
  args: [npm, run, test:unit]

---

# from examples/vote-helm/vote/garden.yml
kind: Deploy
description: Helm chart for the voting UI
type: helm
name: vote
dependencies:
  - deploy.api # Dependency references are now more explicit.
variables:
  hostname: vote.${var.baseHostname}
spec:
  defaultTarget:
    kind: Deployment
    name: vote
  sync:
    paths:
      - containerPath: /app/src
        sourcePath: .
        mode: two-way
  values:
    image:
      # We now reference build actions instead of modules here.
      repository: ${action.build.vote-image.outputs.deployment-image-name}
      tag: ${action.build.vote-image.version}
    ingress:
      enabled: true
      paths: [/]
      hosts: ["${var.hostname}"]
    env:
      - name: HOSTNAME
        value: ${var.hostname}

---

kind: Test
name: vote-integ
type: kubernetes-exec
dependencies:
  - deploy.vote
timeout: 60
spec:
  resource:
    kind: Deployment
    name: vote
  args: [npm, run, test:integ]