#percentiles

Performance & Scaling Bug Report

Our dashboard p99 looked fine while users timed out: we were averaging per-pod percentiles

Elena Kovač · Jun 12, 2026 · 7 min read

During a partial outage, our latency dashboard showed p99 at 350 ms while one pod in twelve was taking seconds per request. The panel averaged per-pod p99 values. Percentiles cannot be averaged. How we fixed the query by aggregating histogram buckets first, and what that changed for alerts and SLOs.

Performance & Scaling

p95, p99 or p99.9? Choosing latency percentiles for SLOs, benchmarks and dashboards

Elena Kovač · May 25, 2025 · 6 min read

Every team argues about whether to track p95 or p99. The answer depends on traffic volume, fan-out and what the number is for. A practical guide to choosing percentiles for SLOs, alerts, benchmarks and capacity planning, with the sample-size maths, coordinated omission and how to report results honestly.