Receive an event from GitHub → clone the repository → run some tests → report the results. Those were the good old days of CI (Continuous Integration), a simple service that revolutionized how development teams shipped software.
It made us faster at writing more reliable tests and helped us to catch bugs earlier so that we shipped fewer to production. The term CI became common and the practice spread like fire, helping teams ship software faster and more often.
But with the increasing complexity of Cloud Native systems, our dependence on Continuous Integration tools has expanded far beyond its original scope. The first step of right-sizing the role of CI tools is acceptance: CI is just a script runner.
What Continuous Integration tools have become
Application development has evolved massively over the years. We used to compile our different applications and run them in multiple VMs, while nowadays we build them as different containers and deploy them on Kubernetes. They all depend on each other and need different storages that need to be provisioned. Then Kubernetes, Terraform, Serverless, E2E testing, and Cloud providers authentication entered the room.
You need to make sure all these different components work together as expected at every change to your codebase. You don’t want to manually load your binaries in your VM or to manually apply your Kubernetes manifests in production. You want Continuous Delivery. This was a genuine need for the industry and the people who were better positioned to help you solve this problem, in an increasingly complex landscape, were CI providers like CircleCI, Travis, Jenkins, etc.
So they started to transform, from CI to full-blown software delivery platforms, offering complex pipelines, multiple runner types, composable configuration, integration to every possible platform. They placed themselves in the middle of the software delivery lifecycle, successfully enabling it, automating as much as possible.
In doing so, they also became the biggest bottleneck for faster delivery.
How Continuous Integration is holding you back
In theory, having a tool automating everything you need to build, test and to deploy your system can be a great competitive advantage and can save a lot of time instead of running and managing these kinds of operations yourself.
In practice though, those platforms started as task runners: they started as generic and technology agnostic. They never had any concept of what running a test means, much less the difference between an integration test and an end-to-end test and how they impact a CI run, or any mechanism to define that.
That became a problem as soon as your application grew in complexity and CI service providers tried to solve this problem in many ways: offering custom flexible configuration options that only made it impossible to switch to other providers. They offered hosted runners for when you need to run heavy workloads, but you have to provide infra yourself. They offer secrets and configuration, but with poor tooling around that, trapping you in their ecosystems.
These tools don’t have any insights into how your stack is built, what your services depend on, what parts of your code have changed in-between PRs and what needs to be rebuilt. Sure, they allow you to declare dependencies between steps but maintaining the cognitive load of the actual application level dependencies are left as an exercise to the end user.
What was supposed to be a simple tool for scheduling and running tasks, became a complex monolithic agent, difficult to debug, hardly descriptive of your system, slowing your team down.
The future of CI
I believe teams are starting to realize how complex and bloated those systems have become. We absolutely need the automation currently owned by CI systems, but we need it to de-silo it from CI service providers.
The future of CI requires us to stop for a second and start again from what we were promised at the beginning: simple version control software integration, scheduling and task runner. And we need to seriously take a look at what CI should be doing vs what we gave it control to do.
It’s 2023, I would love to see mature platform and DevOps teams working on the following features:
Portable pipelines
You should be able to run your pipeline regardless of your CI/CD provider of choice. Your pipeline should be able to express the complexity of your application, be dependency-aware and easy-to-configure and edit as your application or stack grows. It should live in your repository and be completely independent and platform agnostic.
Our current provider bumped their prices? I want to grab my bags and set up shop on another platform, as easy and fast as possible. Is the new provider service slow and buggy? I set up an internal platform and bring my pipeline there.
I want to change my pipeline because my application or stack changes, not because my provider has become too costly or because they are not supporting certain platforms anymore.
Debuggable pipelines
You should be able to easily debug your pipelines. We should stop thinking of CI providers as inaccessible black boxes and we should aim at being able to run pipelines from anywhere: your deploy dependency is broken? I don’t want to access the logs from my machine, I want to run the pipeline from my laptop and check the execution logs in real time.
I want to try and debug a new test setup before even pushing it to my repository.
The times of commit, push, wait for the CI to see if the changes are valid, are over.
Composable blocks
Microservices architecture taught us that, in certain scenarios, splitting up your complex domain in smaller subdomains can be beneficial. Why aren’t we doing so with our delivery process? Why are we preferring “smart” all-encompassing solutions over composability? We should strive to describe our system using composable blocks and the dependencies between them.
Leave CI to do what it’s best designed to do: integrate with our VCS, scheduling and executing tasks, and reporting errors.
Conclusion
The CI platform became a delivery platform out of necessity and opportunity, turning into a velocity bottleneck for serious development teams. It’s our job as engineers to sit down and evaluate if the current approach is sustainable in the long run and how much control we want to take back from CI vendors.
Our team at Garden have been thinking about how to tackle these problems for years. If you recognise yourself in the struggle with the complexity of modern-day pipelines, take a look at Garden. With Garden you can codify your vendor agnostic pipelines, run them everywhere and speed up your delivery cycles considerably. Check it out.
This article first appeared in The New Stack as "Continuous Integration is Just a Script Runner."
