The Secret Garden: RBAC and Secrets in Garden Enterprise
By Mike Winters
March 11, 2021
How Garden Enterprise simplifies governance across users & environments, all in one place
Secrets management can be complex and time-consuming, especially in teams with many developers, services, and environments.
But it’s really important to get it right—the security of your applications and infrastructure depends on it.
Garden Enterprise’s RBAC and secrets management features enable secrets-as-a-service so your developers never waste a minute dealing with secrets, and you get peace of mind that only the right people have access to the right systems and hardware.
If you'd like to learn more about Garden Enterprise, please
The problems that development and ops teams spend the most time on often aren’t the problems that have the biggest impact on the business. But they are often problems that are common across organizations of all shapes and sizes.
At Garden, our broader mission is to eliminate as much unnecessary wasted time from the dev process as we can. You know the kind of stuff we’re talking about: manually configuring environments, scrolling through outdated READMEs to make sense of dependencies, waiting on slow tests and builds, troubleshooting mysterious CI errors, and so on.
And so we’re always on the lookout for new categories of unnecessary wasted time that development teams are dealing with.
Having spent a lot of time with users, it’s become clear to us that secrets management (as it’s often implemented) fits squarely in the category of unnecessary wasted time. It’s a surprisingly complex task, and even if you get it right, all you’ve done is fulfill a basic (but important) security requirement. You haven’t shipped anything new. You haven’t made your existing products faster.
That’s why we’re excited to unveil our new RBAC and secrets management capabilities in
. In this post, we’ll explain why secrets management can be so time-consuming, how Garden Enterprise makes it easier, and what this means to the day-to-day development process.
Why are secrets a pain to manage in the first place?
There are two different types of secrets that ops teams have to worry about:
Secret values that the code needs to perform various operations
User credentials needed to access infrastructure
Let’s focus first on #1. An example is a database password that an application needs to be able to talk to a database that's managed by your cloud provider.
The application API needs to be able to call the database, read data from it, and write data to it. The database is password protected, and the API needs this password to be able to access it. So where do you store passwords like this one?
Putting it in your source code is a big no-no—even with a private GitHub repo, you don’t want an access token in your Git history forever.
You could store them in your CI system, but what about development or testing environments that are spun up outside of CI? For many teams, non-CI environments constitute the majority of environments.
You could encrypt the keys in your source code, but this requires all your devs to have access to an encrypt/decrypt tool.
You could set up your own secrets manager from scratch, but then you’ve got a complicated internal product you have to roll out then maintain for the long haul. You’re looking at a ton of manual effort just to be able to keep your business running securely.
In short, there's no simple answer for where to store your keys.
It’s a problem that becomes more difficult with scale, both in terms of more developers and more services. If you’re going to let every dev spin up their own environments on demand—a best practice that’s becoming increasingly common—then in the example we discussed above, each dev will need their own database key. Plus keys for any other systems the application needs to access. And they’ll need a way to store these keys securely and update them occasionally.
And that’s just for development environments. You probably have other environments, too (such as testing and staging), and different application secrets and user permissions for each environment.
It’s a challenge that can get really big, really quickly.
Roles and secrets in Garden Enterprise
Alright, onto the fun stuff. Let’s walk through RBAC and secrets management in Garden Enterprise.
First, there are three things you should know about how RBAC (role-based access control) works in Garden:
It’s possible to create different user groups made up of users with similar access levels (e.g. “Developers”)
Users and groups can be mapped to environments (e.g. the “Developers” group can access everything related to the “development” environment, but not the “staging” environment)
Secrets can be mapped to users (e.g. an SRE or all of your developers) _and _to environments (dev vs. staging vs. prod)
Adding a new team member in Garden Enterprise
Garden Enterprise takes a “Garden-native” approach to secrets that fits perfectly with how Garden works in general. If we were to describe Garden in four words, we’d choose, “configure once, run everywhere.” You can use the same Garden configuration to spin up a development environment, run your CI pipelines, deploy to production, and so on.
When deploying to different environments, you’re likely going to have to account for different services that require different sets of environment variables. For example, you might use a throwaway database in your dev environment and a cloud-managed database for other environments, both of which have their own access keys.
Different environments may require different services. For example, you might use a throwaway database in your dev environment and a cloud-managed database for other environments. Other services may be deployed to all your environments, but require a unique set of environment variables for each one.
Garden has always made it easy to configure environment-specific services. When you run garden deploy, you can simply specify which environment you want to deploy to, and Garden will spin up the services used for that environment. Garden also makes it easy to set different keys for each environment, but you don't want these keys sitting in your Garden configuration file for the reasons we discussed above.
When you store secrets in Garden Enterprise, Garden can simply read the secrets directly when deploying to a given environment. Developers don't have to store any keys on their local machine, and when a key changes, it's no effort for developers to use the new one—there’s no manual updating they need to do. Ops teams can simply take care of it behind the scenes.
Adding a new secret scoped to both an environment and a user
If there are two of the same keys in Garden Enterprise for a single environment, the key that’s scoped to a user will take precedence. If a user-scoped key doesn't exist, then Garden Enterprise will check for the environment-wide keys, then for project-wide keys.
Adding a new secret scoped to an environment only
For example, you can have a single environment variable in your configuration called DATABASE_PASSWORD. In Garden Enterprise, you can assign multiple different values to this variable, depending on the environment and/or the user. This means you don't need to duplicate your configuration or resort to awkward YAML templating depending on who is deploying to what environment. Again: configure once, run everywhere.
Good news: this lets you control your infrastructure, too!
Next, let’s spend some time on #2: user credentials needed to access infrastructure.
Different environments often run in separate infrastructures, too, which means differences in configuration beyond services and environment variables. Once again, Garden has always made it easy to deploy to different infrastructure depending on the environment, but in most cases, organizations also want fine-grained control over who can deploy where.
For instance, you might be OK with every developer deploying freely to your dev cluster, but you probably don’t want every dev to deploy on demand to your staging cluster (and certainly not to your production cluster).
With its environment-level scopes, Garden Enterprise can do more than store secrets needed for application code—it also can store secrets to control who can actually deploy to different infrastructure.
—not only can you automate the provisioning of your infrastructure with every deploy, but you can also be sure no one’s messing with infrastructure they shouldn’t be. Giving developers access to self-service infrastructure provisioning and automating their entire workflow? Great! Letting developers deploy to and spin up infrastructure on every cluster in your organization? Potentially not great!
Because environments and secrets can be scoped to a user in Garden, it’s easy to limit which users can interact with which environments. You don’t need a separate secrets management solution to control access to infrastructure vs. databases and other systems used by your applications—it’s all handled in one place.
It’s easy to imagine how secrets management is especially powerful when combined with RBAC. This role-based approach to secrets makes it easy to onboard and offboard developers, to update keys without any overhead for devs, or to bring on contractors for short-lived products—and to do it all securely and efficiently.
Getting started and next steps
In short, the combination of secrets management and RBAC in Garden Enterprise makes it easy to manage access to application keys and environments all in one place, out of the box.
Rather than taking on the burden of a custom in-house solution (or pushing the burden of dealing with secrets to your developers), Garden Enterprise lets you offer secrets-as-a-service while giving you the fine-grained control you need for secure applications and secure infrastructure.
We’re really excited to have finished this first iteration and to be getting it in the hands of our customers, and there’s still more we plan to build here. At the top of our list are features that will make it possible to create secrets programmatically or to bulk import them, which is especially important for large teams and/or large applications.
If you have questions for us or you’d like to learn more, feel free to