The example mentions cockroachdb, but the old version never mentions how that was deployed but shows the dependency on my-cockroachdb while the new version drops the cockroachdb entirely. I think it would be good to mention how one would express that dependency. Also previously the container module depended on the create-db task but now it’s just a Run. How would I ensure that the test and deploy actually spawn the DB first and then run create-db before?
We’ve just announced a new Garden version, which we’re naming Bonsai! This update is one of the most substantial of the year, as it incorporates a range of new features and enhancements that significantly augment the power and flexibility of Garden
Garden’s philosophy centers around the idea that every step involved in developing and testing an application falls into one of four types, each with its own semantics:
We noticed that almost all dev automation and CI pipelines boil down to a sequence of a few basic types of operations.
- Build
- Deploy
- Run
- Test
With the Bonsai release, we’re adding greater clarity around these actions so that they’re more flexible, more extensible, and fit better into CI pipelines.
Upgrading to Bonsai provides a more flexible and robust workflow which we will briefly describe in this blog post. Please refer to our documentation or our announcement post for more detailed information on Bonsai, including deprecations and new features.
In Bonsai, we’ve changed how we define our Stack Graph by using actions instead of modules to make Garden more powerful and extensible.
With the new action types in Bonsai, you can enjoy more flexibility and customization in your dev automation compared to the old module-based approach. Modules were a more hierarchical solution for building, testing and deploying services, while the new actions are a more lightweight and flexible approach, allowing for deeper customization to fit your specific needs while also mapping more naturally to the way people are used to writing CI pipelines.
With Garden, you codify your stack with YAML declarations that describe how your system is built, deployed, and tested.
If you can’t migrate your modules to actions, we ensure that actions are backward compatible with modules.
Actions can depend on any action, making Garden more flexible. But, unfortunately, modules can’t rely on actions or vice versa, so make sure you consider that when migrating from 0.12 to Bonsai 💬.
Examples
Old example using Modules which we're going to transform into the Actions-driven format
🌑
kind: Module
services:
- name: api
tests:
- name: unit
args: [npm, test]New Actions-driven format
kind: Test
name: api-unit
build: api
type: container
spec:
args: [npm, test]🌑
With Actions, we’ve decoupled your tests, services, and builds into separate actions. This change gives you the ultimate flexibility as now you can define each stage of the SDLC separately, which means less repetitive work and more clarity in the configuration.
This makes it easy to e.g. run unit tests locally before starting a long Docker build, or to use Runs to fetch tools at the start of a CI pipeline.
The kubernetes-exec actions are an exciting addition to Garden'. With the new kubernetes-exec Run and Test action types, you can now run your tests and tasks without rebuilding, making the process faster and more efficient. kubernetes-exec is particularly useful for ad-hoc tasks and running integration or end-to-end tests during development.
Additionally, for Runs (previously known as tasks in version 0.12), you can reduce the time it takes to spin up your stack, allowing for quicker execution of your code. Upgrade to the latest version of Garden and take advantage of this powerful new action to improve your development workflow.
Example
Old test, nested inside a module
New kubernetes-exec type
The new kubernetes-exec action type enables you to specify the resource in which your test or run (previously known as tasks in 0.12) will execute, providing faster tests and allowing users to run tasks in pods that use persistent volumes where creating a new pod for each run is not an option.
In Bonsai, we improved the user experience by adding more verbosity and clarity to Garden’s logs by default. This feature is handy when debugging issues and interacting with the CLI.
In the following action snippet, we set kind to be Deploy2, which is not a valid action type, so we expect this to fail.
Now Garden points us to the exact file where the issue is, making debugging your configuration even easier 🎉.
The new garden dev command allows developers to perform various tasks, such as starting and stopping code syncing for individual deployments and running actions like Tests or Runs. This command seamlessly works with the new kubernetes-exec actions, resulting in a faster and smoother developer experience.
In a short video, we demonstrate how to use the garden deploy --sync command in one terminal and the new interactive garden dev command in another terminal. With this new feature, developers can seamlessly stop/start synchronization, check container logs, and perform other tasks without waiting for Garden to refresh the state of resources and sync back with the cluster.
To ensure a smooth migration from 0.12 to Bonsai, it's essential to carefully prepare for the change by considering potential edge cases. This section will provide recommended preparation steps to guide you through the migration process. Following these steps can minimize the likelihood of encountering issues during the transition.
> **
If you are using deprecated features, we recommend finding a workaround and adjusting your configurations to avoid any issues during the migration process.
- The deprecated hot-reload feature has been removed; use sync instead.
- The deprecated cluster-docker build mode has been removed; use cluster-buildkit or kaniko instead.
- Dropped support for deploying an in-cluster registry.
- Removed support for the following providers: google-app-engine, google-cloud-functions, local-google-cloud-functions, maven-container, npm-package, openfaas.
- Check out this comprehensive walkthrough on deploying ExternalDNS and cert-manager declaratively using Garden! It’s like a turbocharged Helmfile for all your global Helm charts. The cert-manager plugin is now deprecated.
One of the most important steps in a migration is to minimize the risks, preventing any incident caused by migration is one of the best approaches to handling change.
- Make sure to back up your configuration in a safe place to facilitate rolling back in case of an incident.
- Start the upgrade process with smaller scopes to avoid changing too much in a single run. This helps to minimize the risk of encountering issues during the transition. (e.g, start the migration process in a local environment → Dev → Staging → Prod, instead of going directly to production.)
- Review the breaking changes in the Garden CLI tool and take note of any potential issues that may affect your project.
- Keep in mind that while Garden Bonsai is mostly backward-compatible with the old configuration format, you will need to keep an eye on any issues.
Ready to move to Bonsai and take advantage of this amazingly powerful, simple, and flexible new workflow? First, make sure you’re running the latest Bonsai release by typing into your terminal: garden self-update.
To install Bonsai use the following command garden self-update 0.13.0-0
🌑
To start, we base the migration on one of the more common use cases of Garden; a simple container module implementation for a Flask application that uses a CockroachDB database.
Flask application migration
The following configuration deploys a Flask API, connecting to a CockroachDB instance, which you can see is listed as a dependency. We've deployed CockroachDB as a Helm chart. The Flask API has a task that runs a command to create the database but only after CockroachDB has deployed.
kind: Module
type: container
name: flask-app
build:
targetImage: production
services:
- name: api
ports:
- name: http
containerPort: 8000
tasks:
- name: create-db
command: [/bin/sh, -c]
args:
- python manage.py create_db
dependencies:
- my-cockroachdb
tests:
- name: my-test
command: [pytest]
dependencies:
- flask-app
dependencies:
- create-db
---
kind: Module
type: helm
name: my-cockroachdb
repo: https://charts.cockroachdb.com/
chart: cockroachdb
version: 10.0.0
atomicInstall: false
values:
storage:
persistentVolume:
size: "25Gi"
tls:
enabled: false🌑
Notice that all the configuration is nested in the same block, which can get messy quickly as your service/configuration grows.
Actions are one of the main features of Garden Bonsai. These separate modules into actions with small, clean, and scoped blocks of code for each step in your CI pipelines and dev workflows. The process to break your module up into actions is quite simple; instead of having a single block, we will separate our components based on the kind of step we’re looking at: build, deploy, test, and run.
Create a new file called bonsai.garden.yml (this will make it easier to differentiate between our new and old configuration).
Old build
New Build action:
When turning modules into actions, the name is important; we use the name flask-builder because it will serve as the ID we use to define the dependencies between the other components. (Deploy, Test and Run actions).
In Bonsai, the service definition from the old module syntax, becomes its own Deploy action.
🌑
Old service definition:
services:
- name: api
ports:
- name: http
containerPort: 8000Something to note is that in the container module, the service is inside the same block in the container module. Therefore, we must repeat ourselves to create multiple, independent relationships between Modules that re-use this service.
New Deploy action:
---
kind: Deploy
name: api
build: flask-builder
type: container
spec:
ports:
- name: http
containerPort: 8000
ingresses:
- path: /hello-a
port: http
dependencies:
- create-dbAs we go down the example, you’ll notice three dashes --- between actions, this is how we separate the new Action types within the same file. Notice also that flask-builder shows up in this Action, this defines the relationship between both.
We need a second Deploy block because the old Helm module is now a Deploy action of type Helm. The old service definition is now a Deploy action of type container.
---
kind: Deploy
type: helm
name: my-cockroachdb
spec:
atomic: false
chart:
name: cockroachdb
repo: https://charts.cockroachdb.com/
version: 10.0.0
values:
storage:
persistentVolume:
size: "25Gi"
tls:
enabled: falseWe've moved our custom Helm chart values into the spec.values block. This is where you can define any custom values for your Helm chart.
Chart name, repo, and version are now defined in the spec.chart block. You'll notice that top-level configuration of an action block is reserved for universal configuration options that apply to all action types. For example, the name of the action, the build dependency, and the type of action. Configuration of the action itself is now defined in the spec block.
🌑
Here’s the old test. The old test has the same problem as the old service: we’re in-lining all our relationships inside one giant container Module. Not so flexible.
Old test:
tests:
- name: my-test
command: [pytest]
dependencies:
- flask-app🌑
New Test action:
---
kind: Test
name: my-test
build: flask-builder
type: container
spec:
args: [pytest]
dependencies:
- deploy.apiNotice how we define our dependency by its actionable relationship in Bonsai in two parts: the build: flask-builder and dependencies: deploy.api. build: flask-builder sets our container image to the image of our Build action. dependencies: deploy.api ensures our Flask API is up and running before the test runs. This is a lot more flexible than the old way of defining dependencies, as we can now define dependencies on any action, not just the Module itself.
🌑
These new Run and Test action types introduce rebuild-free Tests and Runs which is a huge time-saver.
Finally, here’s the old Task, which required a messy web of inter-dependencies in order to work. 🏗️
🌑
Old Task
tasks:
- name: create-db
command: [/bin/sh, -c]
args:
- python manage.py create_db
dependencies:
- my-cockroachdbThis gets pretty confusing, as we had to define the tasks to create the database, those tasks belonged to its parent container Module, and in order for the task to run, we had to declare an additional dependency inside of the same Module on the task.
In Bonsai, this is a lot clearer:
---
kind: Run
name: create-db
type: container
build: flask-builder
spec:
args: [python manage.py create_db]
dependencies:
- my-cockroachdbTasks are now Run actions. Everything our Run action needs is defined in the same block, and we can define our dependencies across many actions, not just the Module itself. In Bonsai, any action can depend on any other action.
🌑
Now it’s your turn! Open your old Garden module files and make the changes demonstrated for all your services, tasks, builds, and tests.
Here are some of our best tips to make sure you succeed in your migration to Garden Bonsai.
- Carefully plan the migration: outline a strategy that includes the steps, timeline, and necessary resources for your migration. ⏱️
- Test and validate the migration: simulate the migration in a test environment to identify and resolve any potential issues before implementing the migration in production. Additionally, create a rollback plan in case any unforeseen issues arise during the migration process ✅.
- Backup your configuration: prior to migration, ensure that your old configuration is safe in case you need to go back to it at any point, also you might have to compare your configuration during the transition. 📝
- Communicate effectively: inform all stakeholders about the migration process, including timelines, expected impacts, and any necessary changes to their workflows. Provide regular updates and opportunities for feedback to maintain open lines of communication throughout the migration process. 📞
- Consider reaching out to the Garden Bonsai community on Discord for help and guidance. The Community Engineering team can provide valuable support during the migration process.
Upgrading to Garden's Bonsai provides a more flexible and robust workflow that maps to the Software Development Life Cycle. The new features and enhancements provide ultimate flexibility to connect any stage of the SDLC with less repetitive work and more clarity. ✅
If you're interested in learning more about Garden and its many capabilities, make sure to join our vibrant community on Discord. Our experienced developers and Community Engineering team will be there to welcome you and answer any questions you may have. Join us today and discover the future of software development with Garden!