PRODUCTION SCENARIO
An API gateway serves about 70 requests a minute against a prompt in three parts: a 25,000-token knowledge base and system instruction, then a per-request timestamp and session ID, then the user question. The team set cache_control with type ephemeral on the first part. Analytics report a 0% cache hit rate.
What explains the zero hit rate?
Answering here is anonymous. Nothing is saved unless you sign in.
Show answer and explanation
Answer: The timestamp and session ID sit inside the block that carries cache_control
A cache read needs the prefix ahead of the breakpoint to be byte-identical, and a per-request timestamp inside that block changes it every time. The documented fix is to move the breakpoint to the last block that stays the same and let the varying content follow it. The minimum cacheable prefix tops out at 4,096 tokens, so a 25,000-token block clears it comfortably; 70 requests a minute refreshes the default five-minute lifetime many times over; and ephemeral is the only cache type there is, with the one-hour ttl an option for holding entries longer, not a condition for writing them.