Helm without tears: Maximizing developer productivity with Helm Charts and Garden

Tao Hansen

January 16, 2023

Introduction to Helm charts

Love it or hate it, Helm is the ubiquitous package manager for Kubernetes. With Helm you define, install, and upgrade complex Kubernetes applications in “charts”. Charts provide an organized way to bundle Kubernetes manifests as packages with their lifecycle controlled by Helm or other GitOps tools like Argo CD and Flux.

Teams can struggle with Helm because its templating language and imperative mode of operation make it inscrutable and difficult to reason through. Teams instrumentalize Helm with a rat’s nest of tooling that takes weeks of skittering through docs.

In this article, we'll show you how to deploy a Helm chart to Kubernetes using Garden to make the most of automatic tests, dynamic variable injection, easy in-cluster image builds and hot reloading that accelerates your software development lifecycle past anything possible with Helm alone.

Limitations of Helm charts

Helm charts use key-value pairs defined in a values.yaml file to configure charts for specific environments and clusters. These values are simply declared, which makes baking advanced logic such as conditionals a chore taken care of by Go helper templates. These helper templates can be difficult for DevOps Engineers to reason through, especially when template functions are used in abundance. Users of Helm are advised to be aware of these and other limitations before adoption.

Not a standalone solution

Platform teams will orchestrate a veritable suite of complex tooling to “plug the gaps” of Helm or extend it. Helm charts require a Kubernetes cluster to be installed and running in order to be used. Users must first set up and configure a Kubernetes cluster before they can use Helm charts to deploy their applications. Additionally, Helm charts are not designed to manage the underlying infrastructure on which the Kubernetes cluster is running, such as the servers and network components. Users must use other tools and solutions to manage these components.

Flaky lifecycle management

If a Helm chart deployment is canceled, a chart can end up in an inconsistent state, requiring manual intervention. Helm’s reliability has improved over time but it remains a problem even in recent versions.

The benefits of using Helm charts with Garden

We’ve read Helm isn’t a standalone solution and that teams reach for rat’s nests of tooling before they’re productive. That’s where Garden is powerful: it’s a holistic tool that takes as its concern your whole software development lifecycle. With Garden, your entire application stack, even down to the infrastructure, is captured. And with Garden Plugins, a cluster can easily be instantiated with infrastructure as code tools like Terraform and Pulumi.

Here is an overview of some of the key benefits of using Helm charts with Garden that we’ll be exploring much more in depth in this article.

Tasks and tests

With Helm charts and Garden, it is easy to run unit and integration tests without pushing your code first to CI. Garden provides a simple and organized way to define and run tests and tasks, making it easy to validate the functionality and performance of your applications.

Easy management of dev, prod, and CI environments

Garden is a tool for deploying and managing Helm charts in various environments, including development, production, and CI. It uses declarative configuration files to define dependencies and infrastructure for Helm charts, ensuring consistent and reproducible environments that can run portably anywhere, whether that’s local, remocal, or your CI pipeline.

Dev Mode AKA Hot Reload

Garden’s Dev Mode allows you to run your application in the cloud or locally, automatically rebuilding and redeploying your application when you make changes to your source code. This makes it easy to iterate quickly on your application and see the changes in real time.

Easy in-cluster image builds

Garden’s support for in-cluster image builds makes it easy to build and push container images from within a Kubernetes cluster. This eliminates the need to set up and manage a separate build environment with no performance penalty to your local machine. No more hot CPU. Dramatically reduced waiting.

Plugins

Garden provides a plugin system that allows you to extend its functionality with plugins. Plugins like Terraform and Pulumi allow you to define the entire lifecycle of your application stack and its underlying infrastructure.

Dynamic variable injection

With Garden, you can easily inject dynamic variables into your application independent of its values.yaml file. Whatever values you define in Garden's project.yml overlay your base values.yaml and they come with the benefit of easy conditionals and other helper functions.

Setting up Helm charts with Garden

To set up Helm charts with Garden, you will need:

Helm installed: curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash

AND

a pre-existing Helm Chart

a sample Helm Chart created with the aid of helm itself

existing Kubernetes manifests we’ll convert to a Helm Chart with Helmify

You’ll also need a Garden project.yml we’ll generate. It’s useful to keep Garden’s Helm documentation handy too.

If you’ve already gone through my previous article on migrating from Docker to Kubernetes, you already have everything you need to continue.

I use a Dev Container that comes with a Helm chart example and all the tooling listed above on GitHub. Feel free to make use of it if you just want to get to the code and don’t necessarily want to work through the progressive approach of the entire article.

File structure of a Helm chart

A typical Helm Chart will be organized as follows:


$ tree
.
├── Chart.yaml
├── README.md
├── templates
│   ├── deployment.yaml
│   ├── _helpers.tpl
│   ├── ingress.yaml
│   └── service.yaml
└── values.yaml

In this example, the Helm chart is organized into the following files and directories:

Chart.yaml

This file contains metadata about the Helm chart, such as its name, version, and dependencies.

README.md

This file contains instructions and documentation for the Helm chart.

templates

This directory contains the templates for the Kubernetes resources that will be deployed by the Helm chart.

_helpers.tpl

This file contains helper functions and variables that are used by the templates in the templates directory.

deployment.yaml

This file contains the template for the Kubernetes deployment that will be deployed by the Helm chart.

ingress.yaml

This file contains the template for the Kubernetes ingress that will be deployed by the Helm chart.

service.yaml

This file contains the template for the Kubernetes service that will be deployed by the Helm chart.

values.yaml

This file contains the default values for the parameters that can be configured for the Helm chart.

Bring your own Helm Chart

Be sure you’re in the project folder of wherever your chart is contained. If you’re planning to use a chart from the internet, that’s fine too.

Create a Helm Chart from scratch

You can produce a Helm chart skeleton with Helm itself using helm create $CHARTNAME where $CHARTNAME is whatever you’d like to call your chart. Read Helm’s quick start guide for more.

Generate your Helm Chart with Helmify

If you have a pile of K8s manifests around, the first step is to convert these to a Helm Chart with Helmify. Helmify requires you to pass your K8s manifests to helmifyfrom STDIN e.g cat my-app.yaml | helmify mychart Helmify will then generate the chart and save it to the directory mychart.

Creating a Garden project and configuring it for use with Helm charts

Once you have generated your Helm Chart, you can create a Garden project and configure it to use Helm charts. To do this, you will need to create a project.garden.yml file in the root of your project with garden create project --skip-comments.

Let me show you what a minimal working example looks like in garden. This should give you an idea of complexity and time to effective use. The bare minimum Garden needs to work is a project file and a module file:

project.garden.yml


kind: Project
name: flask-cockroachdb-example
providers:
  - name: kubernetes
    context: gke_devrel-348008_europe-north1_autopilot-cluster-1 # the name of my cluster
    deploymentRegistry:
      hostname: "docker.io" # or any other container registry endpoint
      namespace: "worldofgeese" # your user's registry namespace. On Docker Hub this is my username

modules.garden.yml


kind: Module
type: helm
name: flask-app
chartPath: flask-app
include:
  - flask-app
values:
  databaseUser: root
  namespace: ${environment.namespace}

---

kind: Module
type: helm
name: my-cockroachdb
repo: https://charts.cockroachdb.com/
chart: cockroachdb
version: 10.0.0
atomicInstall: false
values:
  storage:
    persistentVolume:
      size: "25Gi"

You can generate skeleton files with defaults by invoking garden create project and garden create module respectively. As of version 0.12.48 Garden adds a lot of comments to these files, which can make a user think they’re more complicated than they really are. Projects define things like the types of Kubernetes clusters you have, their contexts, your environments (dev, staging, prod), and image registries. Here’s the project.garden.yml generated by garden create project --skip-comments:


kind: Project
name: $PROJECT_NAME
providers:
  - name: local-kubernetes

If you’re running a local Kubernetes cluster, like Rancher Desktop, you wouldn’t need to make any changes provided your current context was set to Rancher’s cluster. But we’re building against a remote Google Kubernetes Engine (GKE) cluster so we’ll change the local-kubernetes value to kubernetes, the provider for remote Kubernetes clusters. You’ll also notice a deploymentRegistry key required when building images locally for remote targets like our remote cluster. Alternatively, you can harness the power of your cluster to build images for you, discussed later in “In-cluster image builds removes the need for a separate build environment”.


kind: Project
name: flask-cockroachdb-example
providers:
  - name: kubernetes
    context: $YOUR_KUBERNETES_CLUSTER_CONTEXT
    deploymentRegistry:
      hostname: "docker.io" # or any other container registry endpoint
      namespace: "worldofgeese" # your user's registry namespace. On Docker Hub this is my username

If you’re developing microservices, you likely have subfolders in your git repository for each, like api, web, etc. Your module files would ordinarily go there. Garden is smart enough to walk your directory structure and find these module files. Alternatively, you can define all of them at the root level like our project.garden.yml, which is what I’ve done above in our modules.garden.yml file.

You’ll find more examples in our docs when referencing external charts. We also have a section for configuring modules.

Using Garden in your inner development loop

Garden provides a number of features that can be used in your inner development loop with Helm charts. Here are some of the key ways that you can use Garden to accelerate your development process.

Using Helm charts in your Garden project

Once you have completed these steps, you can use Garden to deploy your Helm Chart in a Kubernetes cluster with garden deployl. Garden will automatically install and configure the Helm Chart and its dependencies, allowing you to easily deploy your application to Kubernetes.

Manage dev, prod, and CI environments using Garden

Here’s three powerful additions Garden brings to managing your deployment environments:

Streamline the process of deploying and managing your Helm chart: Garden provides a simple and flexible CLI tool for deploying and managing your Helm chart in different environments, including development, production, and CI. Just target your environment with a simple garden deploy --env dev!
Provide consistent and reproducible environments: Garden uses declarative configuration files to define the dependencies and infrastructure for your Helm chart, ensuring that your environments are consistently configured and reproducible.
Simplify collaboration and testing: Garden’s Stack Graph is portable so you can run it anywhere, even in your CI like GitHub Actions and CircleCI. Garden’s 10,000 foot view of all your dependencies brings cached tests and cluster image builds to your whole team with 1-click preview environments and hosted secrets available in Garden Cloud.

From our minimal Project example above we’ll add a defaultEnvironment and username variable, which together will determine our default namespace demarcated by our username. Feel free to change the username variable for your own. The hostname we expect to reach our service at in each environment are optional.

project.garden.yml


defaultEnvironment: dev
variables:
  username: worldofgeese

environments:
  - name: dev
    defaultNamespace: flask-cockroachdb-example-${variables.username}
    variables:
      hostname: dev.worldofgeese.devrel.demo.garden
  - name: prod
    defaultNamespace: flask-cockroachdb-example
    production: true
    variables:
      hostname: worldofgeese.devrel.demo.garden

  - name: kubernetes
    context: $YOUR_KUBERNETES_CLUSTER_CONTEXT
    namespace: ${environment.namespace}
    defaultHostname: dev.worldofgeese.devrel.demo.garden
    deploymentRegistry:
      hostname: "docker.io"
      # Your user's registry namespace. On Docker Hub this is my username.
      namespace: "worldofgeese"

Our biggest change to our modules is the addition of a new container type module that will take care of dependently building a new image whenever a change is made to source code. Because we’ve set build and values.image in the helm module, our Helm chart will automatically source our image and deploy the freshest code to our cluster. Notice how I’ve made use of variables at the module level: Garden can pass outputs like these among any number of modules or projects. With variable templating like this, you can set environment specific variables as I’ve done in flask-app-image and setting the FLASK_DEBUG variable appropriately in our flask-app Helm chart.

modules.garden.yml


kind: Module
type: helm
name: flask-app
releaseName: flask-app-${variables.username}
chartPath: flask-app
include:
  - flask-app
build:
  dependencies: [flask-cockroachdb-image]
values:
  databaseUser: root
  namespace: ${environment.namespace}
  image:
    repository: ${modules.flask-app-image.outputs.deployment-image-name}
    tag: ${modules.flask-app-image.version}
  env:
    FLASK_DEBUG: "${environment.name == 'prod' ? 0 : 1}"
dependencies:
  - my-cockroachdb

---

kind: Module
type: container
name: flask-app-image
exclude:
  - flask-app
dockerfile: "${environment.name == 'dev' ? 'Dockerfile.dev' : 'Dockerfile.prod'}"
image: flask-app

If you’re interested in bringing Garden into your CI workflow, have a read of our docs where we cover using Garden in CI.

Easily set up and tear down your database using Garden Tasks

One of the key benefits of using Helm charts in your Garden project is the ability to run Tasks. With Garden, you can define tasks that can be run to perform common operations before a service such as setting up a database, seeding it with data, and tearing it down. This makes it easy to quickly set up and reset your database during development, allowing you to iterate quickly and test your application in different scenarios.

Using our example local Helm chart, we can create and seed our database by including two commands, as Tasks:


kind: Module
type: helm
name: flask-app
releaseName: flask-app-${variables.username}
chartPath: flask-app
include:
  - flask-app
build:
  dependencies: [flask-app-image]
values:
  databaseUser: root
  namespace: ${environment.namespace}
  image:
    repository: ${modules.flask-app-image.outputs.deployment-image-name}
    tag: ${modules.flask-app-image.version}
  env:
    FLASK_DEBUG: "${environment.name == 'prod' ? 0 : 1}"
serviceResource:
  kind: Deployment
  name: web
tasks:
  - name: create-db
    command: [/bin/sh, -c]
    args:
      - python manage.py create_db
    dependencies:
      - my-cockroachdb
  - name: seed-db
    command: [/bin/sh, -c]
    args:
      - python manage.py seed_db
    dependencies:
      - create-db
dependencies:
  - seed-db

You’ll notice a few new fields here, including serviceResource and tasks and some new dependencies. Garden Tasks require a serviceResource because Tasks are just pods with a very specific mandate. They go out and do their thing then violently rupture into the Æther. Another helpful comparison is to Kubernetes’s Jobs resource. The serviceResource is identified by a Deployment, DaemontSet, StatefulSet or Pod resource. Because I know my Flask app Deployment named “web” has all the programs needed to run my Tasks, I set it as the serviceResource.

The dependencies are a bit tricky and worth explaining: Tasks only run automatically when they are depended on by other tasks or modules. I can’t list my dependencies in order like this:


tasks:
  - name: create-db
    command: [/bin/sh, -c]
    args:
      - python manage.py create_db
  - name: seed-db
    command: [/bin/sh, -c]
    args:
      - python manage.py seed_db
dependencies:
  - my-cockroachdb
  - create-db
  - seed-db

I need to depend on the tasks in reverse order to ensure they all fire correctly. So the seed-db task is depended on at the Module level, then create-db, the first task I want to fire, becomes a nested dependency of seed-db and itself depends on my-cockroachdb. This is a bit of a gotcha, but it’s worth it for the flexibility it gives you in your workflow.

Use Dev Mode to rapidly iterate

Another key feature of Garden is its support for dev mode. When you enable dev mode with garden dev, Garden will automatically rebuild and redeploy your application when you make changes to your source code. This allows you to quickly see the effects of your changes in real time, without having to manually rebuild and redeploy your application.

To our example with support for Garden Tasks, we’ll now add support for Dev Mode:


kind: Module
type: helm
name: flask-app
releaseName: flask-app-${variables.username}
chartPath: flask-app
include:
  - flask-app
build:
  dependencies: [flask-app-image]
values:
  databaseUser: root
  namespace: ${environment.namespace}
  image:
    repository: ${modules.flask-app-image.outputs.deployment-image-name}
    tag: ${modules.flask-app-image.version}
  env:
    FLASK_DEBUG: "${environment.name == 'prod' ? 0 : 1}"
serviceResource:
  kind: Deployment
  name: web
tasks:
  - name: create-db
    command: [/bin/sh, -c]
    args:
      - python manage.py create_db
    dependencies:
      - my-cockroachdb
  - name: seed-db
    command: [/bin/sh, -c]
    args:
      - python manage.py seed_db
    dependencies:
      - create-db
devMode:
  command: [flask, run, --host=0.0.0.0]
  sync:
    - source: web/
      target: /usr/src/app
dependencies:
  - seed-db

Under devMode is command, which rewrites our Helm chart’s default command with our own. Under sync we define our local source and remote target. I’ve defined the same file paths as in my Dockerfile.dev. Garden will watch live for any changes to code in this source directory and sync it over to my remote container. Pretty neat, right?

You’ll notice a long time to run some of these initial tasks. If I run it again, because Garden caches everything (including Tasks), Garden already sees is as run and skips it. Notice too that a link to my Garden dashboard is given. In VS Code, open this link with a ctrl-click. You may see VS Code offer to open the detected port for you: don’t do this as it drops the key portion of the link, which is required for you to visit the dashboard.

Using Garden in your inner development loop with Helm charts can save you time and effort when developing applications for Kubernetes. Garden’s support for tasks and dev mode makes it easy to set up and manage your development environment, allowing you to focus on building and iterating on your application.

Test your code as you write with Garden Tests

With Garden Tests, you can easily test your code as you write it. Simply define your tests in a tests block and Garden will run them automatically whenever you deploy or, if you’re in Dev Mode, as you write and save your code. This allows you to catch errors and bugs early on. So why wait until the end of your development process or CI to test your code? That’s time spent waiting when you could have been in flow.

I’ll add tests at the same top-level of my flask-app module as tasks. To my example (or your own), format your block like so:


tests:
  - name: my-test
    command: [pytest]
    dependencies:
      - flask-app

Tests in Garden are similar to tasks in that they run a short-lived pod that runs only the command you’ve provided before terminating. I’ve set the dependency to its parent module so it runs after my Flask app has run its own dependency cycle block (waiting for the database to be started and the seed-db task to complete). Now, using garden dev, any tests defined will run automatically, and cached. If you want to run Dev Mode without automatic tests use garden deploy --dev: you can still run all test by running garden test or you can run any test by name e.g. garden test my-test.

Building and deploying images automatically using Garden

Building and deploying images automatically using Garden is another advantage Garden brings to your Helm charts. As we saw earlier, we can tie a container module to our helm module and Garden will rebuild and push that new image with our Helm chart whenever it changes.

There’s a second big advantage: Garden supports in-cluster image building, which means that you can build and push Docker images from within your Kubernetes cluster. This eliminates the need to set up and manage a separate build environment, making it faster and easier to build and deploy your applications. This has the additional benefit of saving time pushing from your local machine to a registry, then pulling from the registry to your cluster because builds are cached locally and in your remote registry. To set this up take the kubernetes block from our project.garden.yml and append buildMode: cluster-buildkit:


  - name: kubernetes
    context: ${providers.terraform.outputs.cluster_name}
    kubeconfig: ${var.terraformInitRoot}/${providers.terraform.outputs.kubeconfig_path}
    namespace: ${environment.namespace}
    defaultHostname: dev.worldofgeese.devrel.demo.garden
    deploymentRegistry:
      hostname: "docker.io"
      # Your user's registry namespace. On Docker Hub this is my username.
      namespace: "worldofgeese" 
    buildMode: cluster-buildkit
    clusterBuildkit:
      rootless: true

Garden offers the option to run BuildKit rootlessly reflected in the configuration above. You’re welcome to give it a try!

If you’re running an Autopilot cluster, in-cluster image builds are not currently available. If you’re running an AKS, EKS or Standard GKE cluster, you’re all set to continue.

I’ve been running docker login when first starting my development environment, which gives my local user rights to push to Docker Hub. We’ll need to do the same but for our cluster so it can pull and push with the same rights as I have. The official Kubernetes docs have a section on transforming local Docker Hub credentials to a Kubernetes Secret. First, docker login, then create the secret from your local credentials:


kubectl create secret generic regcred \
    --from-file=.dockerconfigjson=$HOME/.docker/config.json  \
    --type=kubernetes.io/dockerconfigjson

Then append an imagePullSecrets block to the above like so:


    imagePullSecrets:
      - name: regcred
        namespace: default

Best practices for using Garden in your workflow

There are several general best practices for using Garden in your workflow. These best practices include using Garden tasks to automate common operations, using Garden tests to test before CI, and using Garden’s Dev Mode to quickly iterate on your application. Make good use of Garden’s plugin system to extend its functionality, and use Garden’s in-cluster image building to build and deploy your Docker images and save your laptop’s resources for running the things that really matter, like Slack.

Conclusion

Cleanup environment

It couldn’t be easier to destroy your environment when you’re finished. Just run garden delete environment and Garden will clean up all the resources it created. If you’re using a cloud provider, you’ll also need to delete any resources you created manually, like the Artifact Registry repository. Deleting stacks of services like these with one command is a boon to teams looking to quickly spin up and tear down environments. It’s also a great way to keep your cloud provider costs down.

Recap of the benefits of using Helm charts with Garden

Using Helm charts with Garden provides several benefits for deploying and managing applications in Kubernetes. Some of the key benefits include the ability to easily run tests and tasks, support for hot reload (Dev Mode), easy in-cluster image builds, an ever-expanding plugin ecosystem, and easy management of environments. Using Garden in your inner development loop with Helm charts can save you time and effort when developing applications for Kubernetes, and following best practices can help you get the most out of these tools.

If you’re interested in learning more about how to use Garden with Helm charts, check out our Helm guide and Helm reference. If you’re interested in learning more about how to use Garden with Pulumi, check out our Pulumi guide and Pulumi reference.

Join our community on Discord to ask questions, share your experiences, and get help from the Garden team and other Garden users. Your author publishes The Inner Loop, a newsletter on free and open source software monthly. It’s only good things I love and use in my day to day. You can also follow me on Mastodon.

‍

[Heading image: Ohara Koson, "Suichu no Kakucho" (Wild Geese) (1928). Credit: New Bedford Whaling Museum ]