PRODUCTION SCENARIO
A retailer classifies product returns by using a large model on Amazon Bedrock. Accuracy is acceptable, but inference cost and latency are too high for peak season. The retailer has thousands of stored production prompts and no labeled answers, and needs to serve all peak-season traffic from one smaller model.
Which approach will meet these requirements?
Answering here is anonymous. Nothing is saved unless you sign in.
Show answer and explanation
Answer: Model distillation
Distillation transfers knowledge from a large teacher model to a smaller, faster student: Amazon Bedrock generates teacher responses for the retailer's own prompts and fine-tunes the student on them, so no labeled answers are needed. Supervised fine-tuning needs the labels the retailer lacks, prompt caching only reuses repeated prompt prefixes, and intelligent prompt routing keeps both models in service.