Mean latency hiding a slow tail of twelve-second requests
FDE-AI
Latency, Token Economics & Batch Systems
Medium
Question 17 of 24
Most assistant requests take one second, but one in ten takes twelve seconds. The team's dashboard reports mean latency, currently 2.1 seconds, and users keep complaining about long waits.
Which additional measure makes the slow tail visible?
Show answer and explanation
DThe 95th-percentile latency, split by pipeline stage
The 95th percentile lands inside the slow decile, so it reports what the affected tenth of users actually waits, and splitting it by stage says which stage produced the wait. The standard deviation says the distribution is wide without saying how many requests are slow. The median is one second here and hides the tail completely. The slowest request each hour is a single sample driven by one outlier, so it moves for reasons unrelated to user experience.
Anonymous, no account needed. Your tally stays in this browser.