Tag See all →

#load-balancing

2 stories

Networking & DNS Bug Report

Scaled to 10 pods, one pod at 100% CPU: gRPC connections pinned behind a ClusterIP Service

Priya Raghavan · Sep 29, 2026 · 7 min read

Our pricing service scaled from 3 to 10 replicas under load, but one pod stayed at 100% CPU while others idled. gRPC uses long-lived HTTP/2 connections, and a ClusterIP Service balances connections, not requests. How we found it and fixed it with client-side load balancing, a headless Service and MaxConnectionAge.

Banner: a sudden traffic spike spread by a load balancer across a growing pool of servers

Networking & DNS

Surviving traffic spikes: load balancing algorithms, autoscaling lag and load shedding

John Nguyen · Mar 24, 2025 · 8 min read

Our flash sales pushed traffic to eight times normal in under a minute, and autoscaling arrived five minutes late. This guide covers what actually kept us up: the right balancing algorithm, pre-scaling around known events, queues for deferrable work and shedding load on purpose.