PRODUCTION SCENARIO
A law firm's assistant lets a user upload a 60-page contract and then ask a series of questions about it. Every question resends the full contract to the model. Each answer takes several seconds and consumes thousands of input tokens.
Which approach will reduce both the latency and the input cost of the follow-up questions?
Answering here is anonymous. Nothing is saved unless you sign in.
Show answer and explanation
Answer: Prompt caching
Prompt caching keeps a static prompt prefix, such as an uploaded document, available across requests. Cached tokens are not reprocessed, which cuts latency, and they are billed at a lower cache-read rate. Batch inference is not interactive, and neither a larger context window nor another Region avoids re-reading the contract on every question.