2019 fashion brings neon colors, cargo pants, and auto re-building & re-deploying microservices. Get with the times!
In this year’s dotGo I presented <span class="p-color-bg">go3dprint</span>, a demo project for playing with 3D objects in Go. You can watch it here.
The version presented had zero frills. The code compiled into a single binary, ran on my OS, and the editor simply refreshed the output files. The audience could follow along with zero distractions.
But what if we wanna make it fancy? 🧐🍸
What if we want it, say, cloud-native?
See? Fancy!
What do I mean by that? I want:
- Loosely coupled microservices;
- Running as lightweight containers.
And y’know what, it’s 2019, and I demand:
- Re-builds, re-deploys, and re-tests on every code change, like it’s 2010;
- To be able to change files in live containers, so they can be updated without restarting.
With the right combination of refactoring and tooling, it’s easy peasy.
For the first list, we’ll make changes to our code. For the second, we’ll use Garden.
Let’s get started!
Project Overview
As mentioned, the goal is to play around with 3D in Go. For that, we need three parts:
- A way to generate 2D & 3D objects. We’ll use the sdfx library for this.
- A way to shine light on these objects so that we can see them. We’ll use fauxgl here.
- A way to visualize the output that auto-reloads, so you can see it update as you code.
In the simple version, functionality is split into the files mesh.go and render.go. And to visualize the results we’d simply open the output files on VSCode. It’ll reload them whenever they change.
(One issue is that VSCode treats SVG files as code, not graphics, so I’ve added <span class="p-color-bg">svgtopng.go</span> to convert the SVG files into PNGs.)
We use entr so that the project automatically recompiles on file changes. We can run it as <span class="font-mono">ls *.go | entr -r reload.sh</span>. On every change, it’ll re-run <span class="font-mono">reload.sh</span>, which re-builds and re-runs our project.
Alright, it’s cloud time — let’s get off the ground.
Breaking Up The Monolith
First let’s break things up into services, and split them by functionality. We’ll have a <span class="p-color-bg">mesh</span> service with the contents of our <span class="p-color-bg">mesh.go</span> file, and a similar <span class="p-color-bg">render</span> service.
We can’t use VSCode for visualization anymore, since things won’t be running locally. So let’s add another service and call it <span class="p-color-bg">web</span>.
(We can skip <span class="p-color-bg">svgtopng</span> altogether. Browsers can handle SVG natively.)
Your file tree should look something like this now:
Now for the next layer: Docker. Our services are gonna run in containers, so they need Dockerfiles.
Let’s not get carried away here and stick to functional and simple. Here’s a base image we’re using:
You’ll need a Dockerfile in each module’s folder: mesh, render, and web. This is the base image for them.
Setting Up Garden
Now you’ll be able to use Docker to start every service manually. That’s a lot of work though — don’t.
Let’s use a tool that does all of that for us, and then some: Garden. (See: installation instructions.)
Once installed, setting it up is pretty simple. There’s a tiny project-level config, and then you just jot down how to get each of your services running.
So for the project config, let’s go with:
The project config goes in the project root.
And for our services, let’s add something like this to each of them:
Module configs go in the module’s folder, e.g. mesh/garden.yml
We’ll get this all up and running in just a moment. Remember how we needed to do some re-cloud-native-factoring to our code, though? Let’s do that first.
Bye Function Calls, Hello HTTP
When moving to a micro-service architecture, the main change to your preexisting code is in how communication works between the different parts of your system. In monoliths, that’s mostly function calls. In microservices, it’s HTTP requests.
That means every service needs to listen on HTTP, and function returns need to be converted into HTTP responses.
So every service’s main function should look something like:
<span class="p-color-bg">myFunctionName</span> refers to the function in the code that’ll take care of the HTTP request.
It must have the signature of a HandleFunc. Here’s an example:
Give all your services a fake HandleFunc like the one above, and you’ll be able to build and deploy the whole thing with Garden.
Just run <span class="p-color-bg">garden dev</span> on the project tree, then test it with <span class="p-color-bg">garden call mesh</span> for example.
Cool, right?
For Realsies Now
In the local version, the different parts of the project share data in a very simple way: function <span class="p-color-bg">mesh()</span> saves a file, function <span class="p-color-bg">render()</span> opens it. Now we’ll need to move this data via HTTP.
(Ideally we’d have <span class="p-color-bg">mesh</span> send the data directly, and skip saving the file. I couldn’t find how to do it with this library — let’s roll with it 🤷)
To achieve this flow, the <span class="p-color-bg">web</span> service will get data from one service, POST it to the other, and show the user the result.
To adapt our <span class="p-color-bg">mesh</span> service to this, for example, we have to:
- Re-open the files generated by the <span class="p-color-bg">RenderSVG</span> and <span class="p-color-bg">RenderSTL</span> functions.
- Wrap them in a structure we can send off via HTTP.
- And send it!
This is the gist of things — I’m skipping some steps for brevity. Do check out the source code for the complete solution.
Live Feedback
It really bugs me when I’m writing code and every time I change a tiny teeny thing, I have to tab into my terminal, stop the old process, re-build, re-deploy, re-test, and then check my results.
Garden automates all of this by default. I just save my changes and it does the whole dance for me:
You might notice though that this kills the container, builds a new one, and re-deploys it — all of which take some time.
Garden can also do hot-reloading: changing files inside a running container without killing it.
I find this most useful when working on frontends, where changing an HTML file shouldn’t warrant re-building the whole thing; or when working with languages like Node, where e.g. <span class="p-color-bg">nodemon</span> will re-build the project for me inside the running container.
The way I set up the project for Go is, the service will be re-compiled in my local machine, then the Go binary will be hot-reloaded into the cluster. It’s slightly more complicated than the typical hot-reload set up, so check out the source code for more details.
You can give this a try by running <span class="p-color-bg">garden dev --hot=mesh</span> making changes to the code inside <span class="p-color-bg">mesh-src/</span>.
Pretty slick, right?
In Closing
This article has been abridged for brevity, of course, so do check out the repository for the full version.
Once this migration is complete, we can simply start the Garden console and focus on our code. Any changes will be re-built and re-deployed automatically. This way you can focus on what you want and just ignore everything else.
Who said microservices can’t be fun to work with? :)
I hope you had a fun time with this article. Feel free to get in touch on Slack, and let us know if you need help setting up Garden for your own project!
