Garden & Terraform: Your End-to-End Kubernetes Workflow

By Eythor Magnusson

September 11, 2019

End-to-end terraforming.

We’re proud to announce that Garden now integrates with Terraform — an open-source tool that enables users to create, change, and improve infrastructure.

With the Terraform plugin, Garden can ensure that your infrastructure is provisioned and up to date before deploying the rest of your stack.

How does it work?

Garden is built around a core concept: A pluggable Stack Graph. The Stack Graph is a graph data structure where each node represents an action — build, deploy, run, test — and the edges between them represent dependencies.

This allows you to codify information like service A must be deployed before service B.

Thanks to the Stack Graph, Garden knows that it can build both services in parallel but that it must then queue the deployment of service B until service A is ready and responding to health checks.

Pluggability means that Garden can ask different plugins to execute these actions.

With these two powerful concepts, it’s straightforward to integrate Garden with a tool like Terraform. Garden ensures that the right action gets executed by the right plugin at the right time.

Or put differently, Garden ensures that the Terraform plugin provisions your infrastructure before the Kubernetes plugin deploys your services.

Example

Garden & Terraform example

First we run a command to initialise the environment. This particular example creates a GKE cluster using Terraform. Next we deploy the project to the cluster we just created. Finally we use kubectl to verify that the project was successfully deployed. (The GIF has been sped up.)

In the example above, we start with all of the code but none of the infrastructure. When we initialise the environment, Garden will read our Terraform configuration files, check whether our infrastructure is up to date and apply the configuration if needed. Once our cluster has been created, we deploy the project as usual.

Another common use case is to use Terraform to provision individual cloud services such as databases or message queues. With the Terraform plugin, these actions become nodes in the Stack Graph.

This allows you to queue the deployment of your own services until the cloud services they depend on are ready. You can also reference the outputs from Terraform inside your Garden configuration files. For example, Terraform can output the URI of a cloud database which you can set as an environment variable in your own services.

A complete workflow that combines the two use cases could look something like this:

When you start working, Garden ensures that you have an up to date Kubernetes cluster. It creates one if it needs to. Outputs from the cluster configuration are shared across your project.

Garden builds all your services and deploys them to the cluster. It also runs tests and tasks such as database migrations. Thanks to the Stack Graph, Garden executes these steps in parallel.

If your stack depends on a cloud service, Garden makes sure to first provision it via the Terraform plugin before deploying any services that depend on it.

You can use the outputs from the Terraform module to share the URI of the cloud service with your own services

With everything up and running, Garden now watches your codebase and efficiently re-builds and re-deploys your services as you develop.

All the heavy lifting in the steps above is executed in the cloud —

so you don’t have to worry about your laptop overheating

Codify, codify, codify

By combining Garden and Terraform, we’ve codified the entire workflow — from setting up the cluster to running the project.

And if it’s codified, it’s portable, shareable, and automatable. By simply toggling a flag, you can use the same configuration:

for development,

for creating staging and QA environments from your CI pipeline,

and even for deploying to production.

Now, I know what you’re thinking: This must take a lot of configuration. Well, if you’re already using Terraform, it kind of just looks like this:

# An example garden.yml file
kind: Project
name: terraform-gke
providers:
  - name: terraform
    autoApply: true
    initRoot: "."
  - name: kubernetes
    kubeconfig: ${providers.terraform.outputs.kubeconfig_path}
    context: gke
    defaultHostname: terraform-gke-${local.username}.dev-2.sys.garden
    buildMode: cluster-docker
---
kind: Module
type: terraform
name: tf
autoApply: true
---
kind: Module
type: container
name: my-container
services:
  - name: my-service
    dependencies: [tf]
    env:
      DATABASE_URI: ${runtime.services.tf.outputs.my-db-uri}

An example of a garden.yml configuration file that uses the Terraform plugin.

Conclusion

We’ve shown how Garden can integrate with Terraform to give you a complete end-to-end workflow. This allows you to:

Fully codify your entire application development cycle

Follow best practices and maintain your infrastructure as code

Empower your teams with self-service infrastructure

Go from source code to running your stack in the cloud in just a few commands

To learn more about Garden’s integration with Terraform, check out our docs or take a look at this example project.

If you have any questions, ping us in the #garden channel on the Kubernetes Slack, we’re always happy to help!

Liked this article? Why not share it.