PRODUCTION SCENARIO
A nightly batch of 20,000 classification requests shares an 8,000-token policy prefix. The team enables prompt caching with the default TTL, but most batch requests still report zero cache reads.
What change is recommended?
Answering here is anonymous. Nothing is saved unless you sign in.
Show answer and explanation
Answer: Use the 1-hour cache duration, since batches can run past the 5-minute default
The batch docs suggest the 1-hour cache duration for batches with shared context because batches can take longer than 5 minutes to process, which lets later requests still hit the entry. Batch size and prefix placement do not extend the entry's lifetime.