PRODUCTION SCENARIO
A team must grade 15,000 generated responses for correctness and faithfulness before each release. Its subject-matter experts can review a few dozen responses at most. The team also wants a written rationale for every score.
Which approach will meet these requirements?
Answering here is anonymous. Nothing is saved unless you sign in.
Show answer and explanation
Answer: A model evaluation job that uses an LLM as a judge
In a judge-model evaluation, Amazon Bedrock uses a second large language model to score another model's responses and to explain how it scored each prompt and response pair. That scales to 15,000 responses, which a human work team cannot, and it produces the rationale the team asked for.