Four engineers, one invoicing product, no QA team. This is the risk-based testing strategy we settled on: integration tests over mocks, contract tests at service edges, a handful of E2E happy paths and test data that does not rot.
Our Flutter app grew from five screens to sixty, and every change touched widgets, API calls and state in the same file. The structure we moved to: feature-first folders, with UI, state, repository and service layers whose dependencies point one way, and how each layer is tested.
A Go pricing service's goroutine count grew by thousands per hour. A helper that raced a slow lookup against a timeout left the lookup goroutine blocked forever on an unbuffered channel nobody would read. How we found it with goroutine profiles and fixed it with context and goleak.
A prompt tweak that looks better on five hand-picked examples can quietly break a tenth of real traffic. This guide shows how to build an eval set from production, choose graders, use LLM-as-judge without fooling yourself and gate changes in CI.
A coding agent implemented a discount rule and wrote tests for it. Everything was green, the review looked fine, and production applied the discount to the wrong orders. The agent had written tests from its own implementation, so they asserted the bug. How we changed the workflow so humans own the acceptance criteria.
Our CI compared LLM summaries against stored snapshots at temperature 0 and failed about one run in five. Temperature 0 is not a determinism guarantee on hosted models. We replaced string snapshots with property checks and a scored eval.
We extracted a webhook signature verifier from three Laravel apps into a public package. The steps from a folder of classes to an installable Composer package: structure, composer.json, service provider and auto-discovery, Testbench tests, a CI matrix across Laravel versions, tags and Packagist.
Our shared Terraform modules broke consumers twice in one quarter. We added terraform test: fast plan-mode tests with mocked providers on every pull request, and a nightly apply-mode suite against a sandbox account. The test layout, what to assert, and what it costs.