PRODUCTION SCENARIO
A publisher runs 40,000 article summaries a day on Amazon Bedrock using on-demand inference. Each request sends a short instruction followed by the full article text, and every article is different. The finance team asks engineering to reduce the model bill without changing which model is used.
Which change will most directly reduce the on-demand charges?
Answering here is anonymous. Nothing is saved unless you sign in.
Show answer and explanation
Answer: Remove unneeded prompt text and lower the output token cap.
On-demand inference is billed per million input tokens and per million output tokens, so the lever is sending fewer tokens in and generating fewer out. A Provisioned Throughput commitment bills hourly whether or not it is used and does not touch the on-demand rate. Prompt caching only pays off on a long repeated prefix, and here the shared instruction is short while the article, different every time, is nearly the whole prompt. Few-shot examples steer output length but are re-sent as input on all 40,000 calls a day.
Official Source: https://aws.amazon.com/bedrock/pricing/ ↗