Cutting input token cost on a manual sent with every question

MLA-C02 Operating, Monitoring, and Securing ML and AI Solutions Medium Question 14 of 30

A field service app sends a 40,000-token equipment manual to a foundation model on Amazon Bedrock, followed by a technician's short question. Technicians ask several questions about the same manual within a few minutes, and they expect answers in seconds. Input token charges make up most of the monthly bill. The model supports explicit prompt caching.

Which solution will meet these requirements?

Show answer and explanation

CPut the manual first with a cache checkpoint after it, and append each question after the checkpoint

Explicit prompt caching meets the requirements. It is built for long contexts that are reused across queries: static content goes first, a cache checkpoint marks the end of the reusable prefix, and tokens read from the cache are billed at the model's cache-read rate and need not be reprocessed. Each cache hit resets the time to live, which is 5 minutes on many models, so questions minutes apart keep hitting the cache. Batch inference runs asynchronously, so technicians would not get answers in seconds. A question placed before the manual changes the prompt prefix on every request, which causes cache misses. Splitting the manual still sends every token, and each request loses the rest of the manual's context.

Source: docs.aws.amazon.com/bedrock/latest/userguide/prompt-caching…

Anonymous, no account needed. Your tally stays in this browser.

Next question Start 5 timed questions Browse all 30 original MLA-C02 practice questions