A tech startup is less about the actual tech than one might presume. It is as much a conversation with the market. With real people, and their real-world problems.
It’s been quite a year for Garden. We started on our ambitious journey back in 2018, and still today, it feels like we’re just beginning. Our eyes somehow always gaze forward but it’s important to take a breath now and then, observe what we’ve accomplished and learned, before barreling forward once more.
I hope you’ll indulge me as I do just that. But if you want to skip straight down to the product updates below, I won’t be offended—promise.
How far we’ve come
Just under five years ago, as things were going “cloud native” and the DevOps concept was taking hold, an idea started brewing between myself and my co-founders. Although the sophisticated system architectures of top tech companies finally seemed within reach for the rest of the industry, that potential had not been realized.
We saw that the tools and workflows designed for yesterday’s systems won’t work for a more complex and distributed tomorrow. These tools were designed for the early days of the cloud, for VMs, and systems with relatively few components. Most of the complexity in systems used to live in just code but now the automation around our code—processes, builds, deployment, configuration—carries almost as much complexity as the code we write.
Thus our thesis was born: To realize the potential of all this powerful technology and the practice of DevOps, we need to rethink the tools we use. We must break down silos and redraw the dotted lines we’ve grown used to. When you test, you’re not just testing your code, you’re testing the whole thing. That’s why development tooling, CI, CD, building, deploying, testing, must all combine in one cohesive toolchain.
This was the concept behind Garden’s original design, and I take great pride in the fact that it’s held up from day one.
We’ve repeatedly seen teams fashion this sort of tooling for themselves. It’s simply intuitive, not just to us. Our challenge then is to provide that complete experience for teams of all sizes, instead of every company inventing the same solution separately. That’s the mission.
I’m pleased to see that we’re actually getting there. Garden has become a critical part of many teams’ productivity and we’re scaling rapidly to meet the needs of even more teams–growing our team by 3x this year. Our customers and many open-source users—some brave early adopters among them—validate our progress every day.
Where we’re going: Listening and Applying
Out of many lessons learned on this journey, one reverberates the loudest in my little skull these days: A tech startup is less about the actual tech than one might presume. It is as much a conversation with the market. With real people, and their real-world problems. While our fundamentals have held steady from the start, the solution space is so vast that we must continuously learn from our users.
So what have we learned from our users and how are we applying it to our product?
1: The best of local and remote development
Our users have had great success using dev mode to bring fast iteration to remote dev environments. However, there are common use-cases where the local development experience is simply hard to beat.
This is especially true for applications requiring resource-intensive builds, such as complex web frontends, or Java, Scala and Rust backends, which also greatly benefit from persistent local caches. Sometimes, using your powerful dev laptop to iterate on one or two services, but deploying the rest in the remote environment is the best of both worlds.
Local mode runs one or more services on the developer’s machine, giving them a hybrid environment. Using a secure proxy tunnel, these local services can communicate bidirectionally with the remotely running parts of the stack. This gives the fastest possible build speeds and I/O performance, and works perfectly with local caches, IDEs and debuggers.
2: Actionable insights for DevOps
Build and test times tend to slow down as your application evolves. DevOps engineers need the right data at the right time to proactively fix friction points and inefficiencies in their companies’ DevOps workflows. That’s why we’ve been building Stack Analytics into Garden Cloud.
One of Garden’s hidden superpowers is the great amount of neatly structured data generated by simply using it. We’ve been working hard to give our users better access to this data, both in real-time and in aggregate.
Stack Analytics help you observe and analyze trends across all your builds, tests and deployments:
Also in Garden Cloud, we’ve been developing Stack Streams, our name for a fancy but easy-to-use streaming log viewer. See logs for everything going on in your project, side-by-side or interleaved. Understanding the flows between all your builds, deployments and tests has never been easier:
Lastly, we’ve been experimenting with major improvements to the Garden Core CLI log output, after lots of feedback and experience from our users. You’ll get more actionable information at every turn. We’re excited to include these major improvements in the upcoming 0.13 release of Garden Core.
3: Human terminology for human users
At its core, Garden automates four key actions: Build, deploy, run and test. This is after all what you see in the Stack Graph visualization:
However, early design decisions made how you configure Garden unnecessarily complicated.
We included a concept of a module, which is something you build, and it contains some varying number of services, tasks and tests. In the upcoming 0.13 release of Garden Core, we’re doing away with these superfluous terms.
Instead of modules with nested entities, we’ll just have actions that correspond exactly to those four verbs: Build, Deploy, Run and Test. This will make your setup much easier to understand—for new and current users alike. And as a bonus, now any action can depend on any type of action, which has been a common user request.
It’s a massive refactor so it’s taken some time to complete, but we are very excited about the outcome. Oh and don’t worry, we’ll retain compatibility with your current configuration—you’ll have plenty of time to adapt and we’ll help automate the process.
4: Your graph at your fingertips
One frustration we share with our users is in how static our CLI commands are today. To change what you’re working on, say from one service or test suite to another, you’ve thus far needed to exit the Garden CLI and start it again. This takes time—and sometimes more trial and error than we’d like.
So in Garden Cloud, we’re introducing the Command Palette. With a hotkey in the web UI you can now open a spotlight-style command bar and perform any ad-hoc action easily and quickly:
And we’re not reserving this just for Garden Cloud customers. We’re also working to add this functionality into the garden dev CLI command. Choose what exactly you want the command to do, get results faster, and control any automatic behavior (including dev mode and file watch triggers) with an intuitive command line.
5: Automation over abstraction
When we first started work on Garden, we—like many others since—had the urge to simplify how you configure applications for Kubernetes.
In Garden, this took the shape of the container module type, essentially a simplified Docker Compose-esque abstraction that’s converted to the more complex Kubernetes manifests at runtime. While useful as a jumping-off point for Kubernetes development, we’ve learned that someone will always want that one additional flag.
We certainly get it, abstractions are nice and raw Kubernetes deployments are rather cumbersome. But the thing is, if we just keep adding more fields we’ll basically end up with the same complexity in a different non-standard shape.
So going forward, we’ll rather focus on making it easier to generate and work with actual Kubernetes manifests. The container type will remain, for a long while most likely, but we’ll be steering away from recommending its use. A better practice, ultimately, is to bring production manifests and link them cleanly to Garden’s Stack Graph. Your environments should be production-like, after all.
What’s next?
Oh, so many things to do. It’s quite the rabbit hole we’ve ventured down, here at Garden specifically and really this whole industry of ours.
We’re brimming with ideas, but what’s next should really be up to you. We’d love to hear about your challenges and take our direction from there. Please join our community, post on GitHub, reach out directly, whichever you prefer. We want and need to hear from you.
I can’t overstate how grateful I am to every supporter, every user, every customer, every colleague, past and present. It’s a never-ending journey but we enjoy every step, and it’ll be our pleasure to have you all with us as we march on.
