Our vLLM deployment had an HPA targeting 70 percent CPU. Under a load spike, requests queued for 40 seconds while CPU sat at 15 percent, so nothing scaled. GPU-bound servers need to scale on queue depth. We switched to KEDA on vLLM's waiting-requests metric and fixed warm-up.
Every evening our activity consumer fell 1.8 million records behind and needed two hours to catch up. Adding pods did nothing. The cause was a per-record HTTP call and single-row insert, capped by twelve partitions. Batching and keyed parallelism cut processing cost 20x.
A new KEDA ScaledObject was meant to scale workers from zero on SQS queue depth. The queue reached 40,000 messages and the Deployment stayed at zero. The KEDA operator, not the worker, needed permission to read the queue. How we found it in the ScaledObject status and fixed it with EKS Pod Identity.