PRODUCTION SCENARIO
A publisher runs 40,000 article summaries a day on Amazon Bedrock using on-demand inference. The finance team asks engineering to reduce the model bill without changing which model is used.
Which change will most directly reduce the on-demand charges?
Answering here is anonymous. Nothing is saved unless you sign in.
Show answer and explanation
Answer: Remove unneeded prompt text and cap the output length.
On-demand inference is billed per million input tokens and per million output tokens, so the direct lever is sending fewer tokens in and generating fewer tokens out. Temperature changes randomness rather than length, a Provisioned Throughput commitment is billed hourly whether or not it is used, and a larger model usually costs more per token.
Official Source: https://aws.amazon.com/bedrock/pricing/ ↗