PRODUCTION SCENARIO
A request makes two independent lookups taking 400 ms and 700 ms, then a 900 ms generation call. Ignore overhead and assume those durations stay unchanged.
What is the latency after running the lookups concurrently?
Answering here is anonymous. Nothing is saved unless you sign in.
Show answer and explanation
Answer: 1,600 ms
Concurrent lookups take the maximum of 400 and 700 ms, and generation then adds 900 ms, giving 1,600 ms. 2,000 ms sums all three and is the serial baseline the change was meant to beat. 1,300 ms treats the 400 ms lookup as the prerequisite, but the slower branch gates the join. 900 ms assumes retrieval overlaps generation, which it cannot, since the generator needs both results before it starts.
Official Source: https://developers.openai.com/api/docs/guides/latency-optimization ↗