Migrating to Garden Bonsai: Step-by-Step Guide with Expert Tips

Jhan "Shanky" Silva

May 22, 2023

We’ve just announced a new Garden version, which we’re naming Bonsai! This update is one of the most substantial of the year, as it incorporates a range of new features and enhancements that significantly augment the power and flexibility of Garden ✈️.

Garden’s philosophy centers around the idea that every step involved in developing and testing an application falls into one of four types, each with its own semantics:

We noticed that almost all dev automation and CI pipelines boil down to a sequence of a few basic types of operations.

Build
Deploy
Run
Test

With the Bonsai release, we’re adding greater clarity around these actions so that they’re more flexible, more extensible, and fit better into CI pipelines.

Key Features and Improvements in Bonsai 🔑

Upgrading to Bonsai provides a more flexible and robust workflow which we will briefly describe in this blog post. Please refer to our documentation or our announcement post for more detailed information on Bonsai, including deprecations and new features.

Stack Graph v2 (Actions instead of Modules) 🪴

In Bonsai, we’ve changed how we define our Stack Graph by using actions instead of modules to make Garden more powerful and extensible.

With the new action types in Bonsai, you can enjoy more flexibility and customization in your dev automation compared to the old module-based approach. Modules were a more hierarchical solution for building, testing and deploying services, while the new actions are a more lightweight and flexible approach, allowing for deeper customization to fit your specific needs while also mapping more naturally to the way people are used to writing CI pipelines.

With Garden, you codify your stack with YAML declarations that describe how your system is built, deployed, and tested.

If you can’t migrate your modules to actions, we ensure that actions are backward compatible with modules.

Actions can depend on any action, making Garden more flexible. But, unfortunately, modules can’t rely on actions or vice versa, so make sure you consider that when migrating from 0.12 to Bonsai 💬.

Examples

Old example using Modules which we're going to transform into the Actions-driven format


kind: Module
services:
- name: api
tests:
  - name: unit
    args: [npm, test]

New Actions-driven format


kind: Test
name: api-unit
build: api
type: container
spec:
  args: [npm, test]

With Actions, we’ve decoupled your tests, services, and builds into separate actions. This change gives you the ultimate flexibility as now you can define each stage of the SDLC separately, which means less repetitive work and more clarity in the configuration.

This makes it easy to e.g. run unit tests locally before starting a long Docker build, or to use Runs to fetch tools at the start of a CI pipeline.

Run tests and scripts without rebuilding: the brand new kubernetes-exec actions 🚀

The kubernetes-exec actions are an exciting addition to Garden'. With the new kubernetes-exec Run and Test action types, you can now run your tests and tasks without rebuilding, making the process faster and more efficient. kubernetes-exec is particularly useful for ad-hoc tasks and running integration or end-to-end tests during development.

Additionally, for Runs (previously known as tasks in version 0.12), you can reduce the time it takes to spin up your stack, allowing for quicker execution of your code. Upgrade to the latest version of Garden and take advantage of this powerful new action to improve your development workflow.

Example

Old test, nested inside a module


kind: Module
services:
- name: a
tests:
  - name: unit
    args: [npm, test]

New kubernetes-exec type


kind: Test
type: kubernetes-exec
name: unit
spec:
  args: ["npm", "test"]
  resource:
    kind: Deployment
    name: frontend

The new kubernetes-exec action type enables you to specify the resource in which your test or run (previously known as tasks in 0.12) will execute, providing faster tests and allowing users to run tasks in pods that use persistent volumes where creating a new pod for each run is not an option.

Detailed logging information 🔎

In Bonsai, we improved the user experience by adding more verbosity and clarity to Garden’s logs by default. This feature is handy when debugging issues and interacting with the CLI.

In the following action snippet, we set kind to be Deploy2, which is not a valid action type, so we expect this to fail.


---
kind: Deploy2
name: api
build: flask-builder
type: container
spec:
  ports:
    - name: http
      containerPort: 8000
  ingresses:
    - path: /hello-a
      port: http

Now Garden points us to the exact file where the issue is, making debugging your configuration even easier 🎉.


garden deploy
Deploy 🚀

ℹ cloud-dashboard           → Authorizing...
✔ cloud-dashboard           → Done (in 0.7 sec)
ℹ cloud-dashboard           → Initializing Cloud Dashboard...
ℹ garden                    → Running in environment local.garden-bonsai-migration
ℹ cloud-dashboard           → 🌸 Connected to Cloud Dashboard.

✖ Unknown kind Deploy2 in config at frontend/garden.yml

Interactive dev command 🧲

The new garden dev command allows developers to perform various tasks, such as starting and stopping code syncing for individual deployments and running actions like Tests or Runs. This command seamlessly works with the new kubernetes-exec actions, resulting in a faster and smoother developer experience.

In a short video, we demonstrate how to use the garden deploy --sync command in one terminal and the new interactive garden dev command in another terminal. With this new feature, developers can seamlessly stop/start synchronization, check container logs, and perform other tasks without waiting for Garden to refresh the state of resources and sync back with the cluster.

Preparing for the migration 🏋️

To ensure a smooth migration from 0.12 to Bonsai, it's essential to carefully prepare for the change by considering potential edge cases. This section will provide recommended preparation steps to guide you through the migration process. Following these steps can minimize the likelihood of encountering issues during the transition.

Take deprecations into consideration 👋

> **⚠️**: Note that the Bonsai release has deprecated certain old features.

If you are using deprecated features, we recommend finding a workaround and adjusting your configurations to avoid any issues during the migration process.

The deprecated hot-reload feature has been removed; use sync instead.
The deprecated cluster-docker build mode has been removed; use cluster-buildkit or kaniko instead.
Dropped support for deploying an in-cluster registry.
Removed support for the following providers: google-app-engine, google-cloud-functions, local-google-cloud-functions, maven-container, npm-package, openfaas.
Check out this comprehensive walkthrough on deploying ExternalDNS and cert-manager declaratively using Garden! It’s like a turbocharged Helmfile for all your global Helm charts. The cert-manager plugin is now deprecated.

Minimize the risk ❗

One of the most important steps in a migration is to minimize the risks, preventing any incident caused by migration is one of the best approaches to handling change.

Make sure to back up your configuration in a safe place to facilitate rolling back in case of an incident.
Start the upgrade process with smaller scopes to avoid changing too much in a single run. This helps to minimize the risk of encountering issues during the transition. (e.g, start the migration process in a local environment → Dev → Staging → Prod, instead of going directly to production.)
Review the breaking changes in the Garden CLI tool and take note of any potential issues that may affect your project.
Keep in mind that while Garden Bonsai is mostly backward-compatible with the old configuration format, you will need to keep an eye on any issues.

Step-by-Step Migration Process 🪜

Ready to move to Bonsai and take advantage of this amazingly powerful, simple, and flexible new workflow? First, make sure you’re running the latest Bonsai release by typing into your terminal: garden self-update.

To install Bonsai use the following command garden self-update 0.13.0-0

About the example

To start, we base the migration on one of the more common use cases of Garden; a simple container module implementation for a Flask application that uses a CockroachDB database.

The following configuration deploys a Flask API, connecting to a CockroachDB instance, which you can see is listed as a dependency. We've deployed CockroachDB as a Helm chart. The Flask API has a task that runs a command to create the database but only after CockroachDB has deployed.


kind: Module
type: container
name: flask-app
build:
  targetImage: production
services:
  - name: api
    ports:
      - name: http
        containerPort: 8000
tasks:
  - name: create-db
    command: [/bin/sh, -c]
    args:
      - python manage.py create_db
    dependencies:
      - my-cockroachdb
tests:
  - name: my-test
    command: [pytest]
    dependencies:
      - flask-app
dependencies:
  - create-db
  
---

kind: Module
type: helm
name: my-cockroachdb
repo: https://charts.cockroachdb.com/
chart: cockroachdb
version: 10.0.0
atomicInstall: false
values:
  storage:
    persistentVolume:
      size: "25Gi"
  tls:
    enabled: false

Notice that all the configuration is nested in the same block, which can get messy quickly as your service/configuration grows.

Decomposing our module into actions 🔨

Actions are one of the main features of Garden Bonsai. These separate modules into actions with small, clean, and scoped blocks of code for each step in your CI pipelines and dev workflows. The process to break your module up into actions is quite simple; instead of having a single block, we will separate our components based on the kind of step we’re looking at: build, deploy, test, and run.

Build action

Create a new file called bonsai.garden.yml (this will make it easier to differentiate between our new and old configuration).

Old build


kind: Module
type: container
name: flask-app
build:
  targetImage: production

New Build action:


kind: Build
name: flask-builder
type: container
spec:
  targetImage: production

When turning modules into actions, the name is important; we use the name flask-builder because it will serve as the ID we use to define the dependencies between the other components. (Deploy, Test and Run actions).

Deploy action

In Bonsai, the service definition from the old module syntax, becomes its own Deploy action.

Old service definition:


services:
  - name: api
    ports:
      - name: http
        containerPort: 8000

Something to note is that in the container module, the service is inside the same block in the container module. Therefore, we must repeat ourselves to create multiple, independent relationships between Modules that re-use this service.

New Deploy action:


---
kind: Deploy
name: api
build: flask-builder
type: container
spec:
  ports:
    - name: http
      containerPort: 8000
  ingresses:
    - path: /hello-a
      port: http
dependencies:
  - create-db

As we go down the example, you’ll notice three dashes --- between actions, this is how we separate the new Action types within the same file. Notice also that flask-builder shows up in this Action, this defines the relationship between both.

We need a second Deploy block because the old Helm module is now a Deploy action of type: Helm. The old service definition is now a Deploy action of type: container.


---
kind: Deploy
type: helm
name: my-cockroachdb
spec:
  atomic: false
  chart: 
    name: cockroachdb
    repo: https://charts.cockroachdb.com/
    version: 10.0.0
  values:
    storage:
      persistentVolume:
        size: "25Gi"
    tls:
      enabled: false

We've moved our custom Helm chart values into the spec.values block. This is where you can define any custom values for your Helm chart.

Chart name, repo, and version are now defined in the spec.chart block. You'll notice that top-level configuration of an action block is reserved for universal configuration options that apply to all action types. For example, the name of the action, the build dependency, and the type of action. Configuration of the action itself is now defined in the spec block.

‍

Test action

Here’s the old test. The old test has the same problem as the old service: we’re in-lining all our relationships inside one giant container Module. Not so flexible.

Old test:


tests:
  - name: my-test
    command: [pytest]
    dependencies:
      - flask-app

New Test action:


---
kind: Test
name: my-test
build: flask-builder
type: container
spec:
  args: [pytest]
dependencies: 
  - deploy.api

Notice how we define our dependency by its actionable relationship in Bonsai in two parts: the build: flask-builder and dependencies: deploy.api. build: flask-builder sets our container image to the image of our Build action. dependencies: deploy.api ensures our Flask API is up and running before the test runs. This is a lot more flexible than the old way of defining dependencies, as we can now define dependencies on any action, not just the Module itself.

These new Run and Test action types introduce rebuild-free Tests and Runs which is a huge time-saver.

Run action

Finally, here’s the old Task, which required a messy web of inter-dependencies in order to work. 🏗️

Old Task


tasks:
  - name: create-db
    command: [/bin/sh, -c]
    args:
      - python manage.py create_db
    dependencies:
      - my-cockroachdb

This gets pretty confusing, as we had to define the tasks to create the database, those tasks belonged to its parent container Module, and in order for the task to run, we had to declare an additional dependency inside of the same Module on the task.

In Bonsai, this is a lot clearer:


---
kind: Run
name: create-db
type: container
build: flask-builder
spec:
  args: [python manage.py create_db]
dependencies:
  - my-cockroachdb

Tasks are now Run actions. Everything our Run action needs is defined in the same block, and we can define our dependencies across many actions, not just the Module itself. In Bonsai, any action can depend on any other action.

Now it’s your turn! Open your old Garden module files and make the changes demonstrated for all your services, tasks, builds, and tests.

Tips and Best Practices for Smooth Migration 💯

Here are some of our best tips to make sure you succeed in your migration to Garden Bonsai.

Carefully plan the migration: outline a strategy that includes the steps, timeline, and necessary resources for your migration. ⏱️
Test and validate the migration: simulate the migration in a test environment to identify and resolve any potential issues before implementing the migration in production. Additionally, create a rollback plan in case any unforeseen issues arise during the migration process ✅.
Backup your configuration: prior to migration, ensure that your old configuration is safe in case you need to go back to it at any point, also you might have to compare your configuration during the transition. 📝
Communicate effectively: inform all stakeholders about the migration process, including timelines, expected impacts, and any necessary changes to their workflows. Provide regular updates and opportunities for feedback to maintain open lines of communication throughout the migration process. 📞
Consider reaching out to the Garden Bonsai community on Discord for help and guidance. The Community Engineering team can provide valuable support during the migration process.

Conclusion 🔚

Upgrading to Garden's Bonsai provides a more flexible and robust workflow that maps to the Software Development Life Cycle. The new features and enhancements provide ultimate flexibility to connect any stage of the SDLC with less repetitive work and more clarity. ✅

If you're interested in learning more about Garden and its many capabilities, make sure to join our vibrant community on Discord. Our experienced developers and Community Engineering team will be there to welcome you and answer any questions you may have. Join us today and discover the future of software development with Garden!

‍