Mean latency hiding a slow tail of twelve-second requests

FDE-AI Latency, Token Economics & Batch Systems Medium Question 17 of 24

Most assistant requests take one second, but one in ten takes twelve seconds. The team's dashboard reports mean latency, currently 2.1 seconds, and users keep complaining about long waits.

Which additional measure makes the slow tail visible?

Show answer and explanation

DThe 95th-percentile latency, split by pipeline stage

The 95th percentile lands inside the slow decile, so it reports what the affected tenth of users actually waits, and splitting it by stage says which stage produced the wait. The standard deviation says the distribution is wide without saying how many requests are slow. The median is one second here and hides the tail completely. The slowest request each hour is a single sample driven by one outlier, so it moves for reasons unrelated to user experience.

Source: sre.google/sre-book/monitoring-distributed-systems

Anonymous, no account needed. Your tally stays in this browser.

Next question Start 5 timed questions Browse all 24 original FDE-AI practice questions