Our vLLM deployment had an HPA targeting 70 percent CPU. Under a load spike, requests queued for 40 seconds while CPU sat at 15 percent, so nothing scaled. GPU-bound servers need to scale on queue depth. We switched to KEDA on vLLM's waiting-requests metric and fixed warm-up.
Kubernetes can now change a running pod's CPU and memory requests and limits without recreating it. How the resize subresource works, what resizePolicy controls, which changes still restart a container, how it interacts with QoS classes and VPA, and where we use it in production.
A new KEDA ScaledObject was meant to scale workers from zero on SQS queue depth. The queue reached 40,000 messages and the Deployment stayed at zero. The KEDA operator, not the worker, needed permission to read the queue. How we found it in the ScaledObject status and fixed it with EKS Pod Identity.
Requests decide where a pod is scheduled and how much CPU it is guaranteed; limits decide when it is throttled or killed. This guide explains QoS classes, CFS throttling and OOM kills, and shows how we size containers from real metrics with HPA and VPA.
During a traffic spike our checkout service stayed at 3 pods while CPU was pegged. kubectl get hpa showed <unknown>/70% for weeks. A sidecar container added by a platform change had no CPU request, so utilisation could not be computed. Requests on every container fixed it.
During a flash sale our checkout API on ECS Fargate slowed to multi-second responses, yet the service stayed at six tasks. Target tracking on average CPU at 70% never fired: tasks were blocked on I/O, not CPU, and the max capacity was already reached on one service. How we chose better scaling signals.
Our ECS Fargate workers scaled on SQS queue depth. After each burst, scale-in stopped tasks in the middle of 10-minute PDF jobs; messages reappeared and the jobs ran again, sending customers duplicate documents. How we fixed it with task scale-in protection, SIGTERM handling and a visibility timeout that fits the work.
Our flash sales pushed traffic to eight times normal in under a minute, and autoscaling arrived five minutes late. This guide covers what actually kept us up: the right balancing algorithm, pre-scaling around known events, queues for deferrable work and shedding load on purpose.
After upgrading to Karpenter v1 with WhenEmptyOrUnderutilized consolidation, our EKS cluster replaced nodes constantly and long-running pods were evicted every few minutes. How we read Karpenter's events, tuned consolidateAfter and disruption budgets, and protected jobs that must not be interrupted.