PRODUCTION SCENARIO
A user-facing document QA service must answer within seconds and is capped at 2,000,000 input tokens per minute. Every request carries the same 150,000-token handbook plus a short question, and the team is about to request a higher limit.
What is the most effective first step?
Answering here is anonymous. Nothing is saved unless you sign in.
Show answer and explanation
Answer: Cache the handbook, since on most models cache reads do not count toward that limit
The rate limits page explains that only uncached input tokens count toward the input-tokens-per-minute limit for most models, so caching the repeated handbook raises effective throughput without a limit increase. Output tokens are counted under a separate output limit.
Official Source: https://platform.claude.com/docs/en/api/rate-limits ↗