PRODUCTION SCENARIO
A support assistant receives a mix of trivial questions and complex troubleshooting requests. The team sends every request to its most capable model. Analysis shows that 70% of the bill comes from questions a smaller model in the same family could answer well.
Which solution will reduce the cost with the LEAST development effort?
Answering here is anonymous. Nothing is saved unless you sign in.
Show answer and explanation
Answer: Use Amazon Bedrock Intelligent Prompt Routing.
Intelligent prompt routing exposes one endpoint that predicts the response quality of each model in a family for every request and forwards the request to the model with the best combination of quality and cost. It eliminates the orchestration logic that a custom classifier would require.