PRODUCTION SCENARIO
Sable Freight's quoting service sends a 15,000-token tariff document with every request and returns 429 errors at peak against its tokens-per-minute limit. Cost and latency are acceptable; throughput is not.
Which approach raises the number of requests the service can make per minute?
Answering here is anonymous. Nothing is saved unless you sign in.
Show answer and explanation
Answer: Retrieve only the tariff sections each quote needs from a vector store and send those
Cached input tokens still count toward tokens-per-minute limits and caching does not change how those limits are calculated, so the only lever is sending fewer tokens per request. A per-customer prompt_cache_key partitions cache accounting, a previous_response_id chain re-bills every earlier input token on each turn, and batching amortises the tariff document across quotes but still sends it in full while lowering the request count the question asks to raise.
Official Source: https://developers.openai.com/api/docs/guides/prompt-caching ↗