Minimizing the possibility of releasing software with bugs is always important. This is especially crucial in the continuous software development process where code changes are introduced iteratively and often. In this article, you'll focus on one particular aspect of quality assurance: automated integration testing in Kubernetes.
Integration testing is a phase in the software development testing cycle that often takes place after unit testing. While unit testing helps to test individual features of an application, integration testing integrates these features and tests them as a combined set to ensure that the application works together as expected. Integration testing is useful for identifying issues between services, enhancing test coverage, and improving reliability.
In this article, you'll learn how to set up integration tests for a Kubernetes workload using Garden, a DevOps automation tool used for rapid testing and development. This test setup will work both locally on your machine as well as in a CI/CD pipeline.
What is integration testing?
Integration tests often test interdependent functionalities of an application or service, such as calling the database from a backend API or connecting to a worker.
Although individually testing individual features may not reveal any issues, combining modules developed by different programmers can expose defects such as version incompatibility, API or network connection problems, insufficient exception handling, or inadequate logging. Integration testing helps to identify these issues early on.
In addition, integration testing helps guarantee that the system maintains its expected workflows even after introducing new changes. This frees up workers who might have been stuck fixing undetected bugs so they can pursue more productive endeavors.
Integration testing keeps things running smoothly, ensuring that pods are functional and that services and Ingresses respond correctly at the specified hosts, ports, and paths.
Implementing integration tests in Kubernetes
In this tutorial, you're going to learn how to add Integration tests to a Kubernetes project.
Before you begin, you need to complete the following prerequisites:
- Install Garden.
- Install Docker Desktop. You'll need this to run Kubernetes. Once Docker Desktop has been installed, enable Kubernetes on Docker Desktop using this short guide.
- Install Helm, Kubernetes package provider
- Garden requires Kubernetes ingress controller to be properly configured. If in doubt, use a clean installation of Docker Desktop and Garden will take care of deploying and configuring ingress for you.
Once you've completed the prerequisites, you're ready to begin.
Clone the sample application
First, clone the sample application with the following:
Then navigate into the directory with the following:
Deploy the project using Garden:
You should be able to visit the project page at http://food.suggestions.app.garden/. If the page doesn't load because the DNS address can't be found, you need to update your host file.
You can update the host file at <span class="p-color-bg">/etc/hosts</span> by opening it in your preferred editor or via the terminal.
On macOS or Linux, use the following:
On Windows, open Notepad as an administrator. Then open the <span class="p-color-bg">hosts</span> file in the <span class="p-color-bg">C:\Windows\System32\Drivers\etc</span> directory, add the following, and save it:
At this point, you should be able to view the page:
This is a basic food recommendation app that enables users to suggest new dishes for a restaurant's menu by filling out a form. You can interact with it by filling out the form and then clicking See other's suggestions to see more suggestions:
Behind the scenes, the app is dependent on several small microservices:
- The folder <span class="p-color-bg">food</span> contains a Node.js/Vue app running the application's user interface.
- <span class="p-color-bg">api</span> contains the Flask backend API.
- <span class="p-color-bg">postgres</span> contains the service running the PostgreSQL database that stores the food suggestions.
- <span class="p-color-bg">redis</span> contains the service running both a cache and a message broker that sends data from <span class="p-color-bg">api</span> to a Celery worker.
- <span class="p-color-bg">worker</span> is the microservice that contains a Celery worker that pulls data from the Redis queue and persists the food suggestions to the database.
The application is also configured to be deployed to a Kubernetes cluster with the help of Garden. Notice how you don't have to create separate manifest <span class="p-color-bg">yml</span> files to define pods, deployments, services, or ingresses. Instead, you just have to define project level configurations using <span class="p-color-bg">project.garden.yml</span> and actions to execute for each microservice via <span class="p-color-bg">*.garden.yml</span> files.
For example, consider this <span class="p-color-bg">api/garden.yml</span> file:
In this file, two action kinds are defined for the API backend: <span class="p-color-bg">Build</span> and <span class="p-color-bg">Deploy</span>. The first builds the backend container using the corresponding <span class="p-color-bg">Dockerfile</span> and the second deploys the container on Kubernetes.
Adding automated integration tests
Although unit tests can help catch errors within isolated parts of an application, they're not sufficient for detecting all possible errors that can occur between services and the system as a whole. It's crucial to detect these errors before the services are deployed to production. To accomplish this, let’s create integration tests for the backend <span class="p-color-bg">api</span>.
In the <span class="p-color-bg">api/tests/integration_tests</span> directory, create a file called <span class="p-color-bg">test_app_int.py</span> with the following contents:
This integration test has been set up in the <span class="p-color-bg">test_suggestions</span> method to test all four layers of the backend (<span class="p-color-bg">redis</span>, <span class="p-color-bg">worker</span>, <span class="p-color-bg">postgres</span>, and <span class="p-color-bg">api</span>). It tests the creation of food suggestions by the API and makes sure that the food suggestions are stored in the database. This test also indirectly verifies that the worker functionality is working since the worker's job is to store suggestions in the database.
The <span class="p-color-bg">test_suggestions</span> method not only verifies the created food suggestions but also confirms the presence of all the food suggestions in the Redis cache key <span class="p-color-bg">suggestions</span>. This approach indirectly tests the functionality of the Celery scheduled cron job <span class="p-color-bg">add-every-5-second</span> that updates the cache in the worker microservice. Since the integration test in the <span class="p-color-bg">api</span> backend already tests the interactions with other microservices such as <span class="p-color-bg">redis</span>, <span class="p-color-bg">postgres</span>, and <span class="p-color-bg">worker</span>, adding separate integration tests in each of them is unnecessary.
Once ready, you can run each test separately using the commands <span class="p-color-bg">garden test api-unit</span> and <span class="p-color-bg">garden test api-integ</span> which should return output like this:
Alternatively, you can run all tests using simply the command: <span class="p-color-bg">garden test</span>
With the tests out of the way, it's time to review how to take advantage of this functionality to solve some bugs.
Simulate a bug
Now that you've added some automated integration tests, let's introduce a bug in the code of one of the services to see if the integration tests work.
Let's create a scenario where something goes wrong when you start up the Celery worker in the <span class="p-color-bg">worker</span> service. In <span class="p-color-bg">worker/Dockerfile</span>, remove the <span class="p-color-bg">arg "--beat"</span> on line 14 (*ie* change <span class="p-color-bg">CMD ["celery", "-A", "tasks", "worker", "--beat", "--loglevel=INFO"]</span> to <span class="p-color-bg">CMD ["celery", "-A", "tasks", "worker", "--loglevel=INFO"]</span>).
This modification will stop the Celery scheduled cron jobs from executing, which will consequently prevent the Redis cache key <span class="p-color-bg">suggestions</span> from updating with all the created food suggestions present in the database.
Now, see if you can catch the bug by running the following command:
As you can see, the unit tests run without any errors. However, as expected, integration tests fail:
This occurs because the code checks whether the Redis cache was updated with all the food suggestions present in the database.
After setting up your project, you can deploy it effortlessly with <span class="p-color-bg">garden deploy</span> and when finished, all you have to do is remove it using <span class="p-color-bg">garden cleanup deploy</span>. This streamlined process highlights the indispensability of Garden for developers conducting integration tests on Kubernetes. It provides a much-needed solution by creating production-like environments on demand. This valuable feature allows developers to test their services in environments mirroring the actual production settings, leading to more accurate and reliable results.
Additionally, Garden can rebuild services as required. This feature is made possible through its Stack Graph, which identifies and quickly rebuilds or re-tests the necessary microservices.
Moreover, Garden significantly expedites the testing process by providing a remarkable 80 percent acceleration in end-to-end testing. This substantial improvement minimizes wait times, enables faster feedback loops, and facilitates more rapid iteration within your work processes. For more information on how to add tests using Garden, check out the official documentation.
This article provided you with an understanding of automated integration testing and its significance in the software development process. You learned how to incorporate integration testing in a Kubernetes application using Garden, a tool that can be utilized to improve developer productivity.
Garden combines rapid development, testing, and DevOps automation into a single tool. With Garden, you get access to production-like development environments with minimal configuration required, and you’ll have more time to focus on software development. Visit Garden's official documentation to learn more.
Adeyinka Adegbenro is a software engineer based in Lagos, Nigeria, currently at BriteCore. She loves researching and writing in-depth technical content.