PRODUCTION SCENARIO
Every request includes a long stable compliance policy followed by a caller-specific question. Cache reads stay near zero because the team writes a random request ID into the middle of the policy block.
Which change restores prefix reuse?
Answering here is anonymous. Nothing is saved unless you sign in.
Show answer and explanation
Answer: Move the request ID out of the policy block and put it after the breakpoint
A cache entry matches on an exact prefix, so a random ID inside the policy changes the bytes on every request and nothing before the breakpoint is reusable; moving the volatile content after the breakpoint fixes it. Deterministic field ordering is real advice for a different silent invalidator and does nothing to a value that is random by design. A longer TTL genuinely extends an entry's life but cannot make a changed prefix match. And a second breakpoint splits the prompt without making the policy stable, because the ID sits inside it.