Our main pipeline took 34 minutes, and engineers batched changes to avoid waiting, which made reviews worse. We cut it to 14 minutes without new hardware magic: measure the critical path, parallelise, cache the right layers, split tests by timing and build only what changed.
After a framework upgrade, our CDN cache hit ratio for static assets dropped from 97 to 41 percent and origin data transfer doubled. The new session middleware attached Set-Cookie and Cache-Control private to every response, including JavaScript bundles. Scoping middleware fixed it.
An LRU cache is a hash map plus a linked list. A database index is a B-tree. A rate limiter is often a sorted set. When we started reading system components as data structures, capacity questions and odd latency curves became much easier to explain.
We added a semantic cache to our help assistant to cut LLM cost. Within days a user asking how to cancel a subscription got instructions for cancelling an order. The two questions were 0.92 similar. We added exact-match first, scoped keys, intent gating and a measured threshold.
Users were logged out at random, mostly in the afternoon. Sessions and a fast-growing cache shared one Redis with allkeys-lru, so every hourly cache warm-up evicted idle sessions. Separate DB numbers would not have helped; separate instances did.