PRODUCTION SCENARIO
A Foundry assistant answers the user's exact question, but adds a warranty exception absent from the supplied policy. Its answer is rated highly relevant.
Which additional evaluator targets the observed defect?
Answering here is anonymous. Nothing is saved unless you sign in.
Show answer and explanation
Answer: Groundedness against the supplied context
Groundedness measures how well the response aligns with the given context without fabricating content, the precision aspect and exactly the defect here. Response completeness is the recall aspect; it catches an answer that leaves things out, not one that adds them. Retrieval scores the retrieved chunks, which were fine since the answer is relevant. Fluency scores language, and the invented clause reads well.