Shifting Left to Help a Small Team Move Quickly

At Triple Pat there are two of us, collaborating from opposite sides of the earth. This is an early start up, so we have day jobs. We are older company founders, so we have kids and partners. When something breaks it cuts directly into the one critical resource we most lack: time. So we’ve invested heavily in not letting things become broken.

The industry calls catching problems earlier in the development lifecycle “shifting left” and in large companies it can be (and has been) misused as an excuse to shift more work onto developers and lay off valuable operations staff, testers, and writers. For a two-person team the framing is simpler: every bug we catch automatically is time we can spend sleeping or hanging out with our families.

Here’s what that looks like in practice, working from the outermost layer inward.

Linting things that aren’t code

Most teams hopefully lint their application code. We lint everything.

Our CI pipeline runs ShellCheck on every shell script, actionlint on our GitHub Actions workflows, html5validator and lychee on our website, and yamllint on YAML. Our deployment scripts are infrastructure and deserve the same rigor as everything else. Every tool is turned up to maximum strictness, because maximum clarity is important.

Validating before deploying

Our nginx configuration gets validated before it touches a production server. We spin up an ephemeral Docker container with injected /etc/hosts entries, load the nginx config, and verify it parses correctly. If validation fails, the deploy halts. Same for our alloy config! If these configs are wrong in some way (and each can be subtle), our deployment stops before the bad config is ever turned on.

The pattern extends to our serial mirror deployment: we update one mirror at a time, and a failure on any mirror stops the entire pipeline. This means a bad deploy affects at most one mirror instead of all of them. The remaining mirrors keep serving traffic while we investigate.

This isn’t a Flux thing or a k8s thing or even an Ansible thing, it’s just a simple shell script (which, yes, passes shellcheck) that has the -euxo pipefail options all turned on. So if any command errors, the deployment fails.

Testing as part of the Docker build

Our multi-stage Docker builds run the full test suite during the build phase. If tests fail, you don’t get an image. There’s no “build succeeded, but we should probably run tests before deploying” gap.

Combined with 100% test coverage (we’ve written about why we require this), this means a successful docker build is a meaningful signal that the code works in the environment it will be run in practice. If we accidentally rely on a glibc feature that is not part of musl, then we’ll find out here.

Checking HTML before it ships

The website runs html5validator --Werror in CI on the output before it is copied to the server. Broken HTML, invalid attributes, or accessibility issues can all block deployment. Hugo’s own --panicOnWarning flag catches template problems.

For a static site like ours, this might seem like overkill. But we’ve been bitten by broken links and malformed HTML that looked fine in our browser but broke in others. The validator catches what eyeballs miss, and validating every time means touching old templates is a lot less scary.

Making APIs a compile-time concern

Our most impactful shift-left investment is treating OpenAPI specs as contracts, not documentation. We wrote about this in detail previously, but the short version is that our Go tests validate responses against the spec, and chunks of our Swift client are generated from the spec at build time.

API drift between server and client becomes either a test failure or a compile error. It never becomes a production incident.

The compounding effect

None of these techniques are novel. ShellCheck, Docker multi-stage builds, HTML validation, and OpenAPI specs are all well-documented, well-maintained tools. The value isn’t in any single one. It’s in stacking them.

Each layer catches a different category of mistake:

These layers stack like swiss cheese. Each test is imperfect and lets some bad stuff through, but by creating lots of (hopefully independent) layers we can decrease the probability that the holes “line up” and allow an error all the way through.

A bug has to slip past all of these layers to reach production. For a two-person team that’s the difference between shipping confidently and shipping anxiously.

The cost

This isn’t free. Setting up these pipelines takes time, and maintaining them takes ongoing attention. When a linter rule produces false positives, someone has to investigate. When a validation step is too slow, someone has to optimize it.

But the cost of not doing it is higher. The time spent debugging a lint failure is moveable. Production incidents need to be dealt with immediately, but left-shifted pipeline failures aren’t production incidents, they are code or config issues that can be dealt with on a non-emergency basis. This makes it much harder for anything to steal our time.

The real cost is discipline. It’s tempting to skip the nginx validation step when you “know” the config is fine. It’s tempting to disable the HTML validator when it flags something you don’t think matters. Resist. The whole point is that these checks catch things you didn’t think would be a problem. In exchange for the discipline, less time is stolen from you on a surprise basis. You don’t get interrupted when making art with your kid.

What we are adding next

We are actively using and building integration tests that exercise the full stack with the iOS app talking to Go services through real HTTP. This way, we can integration test the client against an ephemeral server that is running the real code, and we can simulate different forms of success and failure by configuring the server to have or not-have a given account or UUID.

TL;DR

If you’re a small team choosing where to invest engineering time, invest in catching problems early. The tools are mostly free, the setup cost is bounded, and the payoff compounds with every deploy. Each time it seemed a little silly to add yet-another-linter or yet-another-check, but we have always been glad we did; Time spent quickly became time saved every time.