Large-model accuracy at small-model speed for claim triage

MLA-C02 ML Model and Foundation Model (FM) Development Medium Question 11 of 30

An insurance company classifies incoming claim descriptions with a large foundation model on Amazon Bedrock. Accuracy meets the target, but latency and inference cost per request are too high for the planned volume. A smaller model in the same family is fast enough, but its accuracy with the same prompt is 11 points lower. Model invocation logging has captured 6 months of the large model's prompts and responses. The company has no budget for manual labeling.

Which solution will meet these requirements?

Show answer and explanation

DRun an Amazon Bedrock Model Distillation job with the large model as teacher and the invocation logs

Model distillation meets the requirements. It transfers knowledge from a larger teacher model to a smaller, faster and cheaper student, and it can build its training data from the invocation logs the company already has, so no one labels anything by hand. Provisioned Throughput buys more capacity for the same large model at an hourly price; the model does not get smaller, so it does not gain the smaller model's speed. Supervised fine-tuning on adjuster-labeled claims needs the manual labeling the company cannot pay for. Routing to the large model when the small model is wrong assumes the application can tell a wrong answer at request time, which it cannot without a label.

Source: docs.aws.amazon.com/bedrock/latest/userguide/model-distilla…

Anonymous, no account needed. Your tally stays in this browser.

Next question Start 5 timed questions Browse all 30 original MLA-C02 practice questions