When our platform team was asked to host open-weight models for three product teams, the inference engine was the easy choice: vLLM, which most teams already used locally. The open question was everything around it. Who creates the pods, who decides how many replicas run, how a 70B model spread across GPUs is scheduled, and how an upgrade rolls out without dropping streams. On Kubernetes the two common answers are KServe, a Kubernetes-native model serving control plane, and KubeRay, an operator...