PRODUCTION SCENARIO
A Generative AI Engineer reviews a RAG assistant in MLflow. Traces show the retriever returning the correct policy chunk on 92% of questions, yet roughly one response in six states something that chunk contradicts.
Which built-in judge should the engineer score the application with?
Answering here is anonymous. Nothing is saved unless you sign in.
Show answer and explanation
Answer: RetrievalGroundedness, which checks the response against the retrieved chunks
RetrievalGroundedness assesses whether the response is factually supported by the provided context, which is the retriever-good, answer-wrong split the traces show, and it needs no ground truth. RetrievalSufficiency asks whether the chunks contain the answer, which the traces already establish, and it does require ground truth. Correctness also needs expected facts and blends retrieval with generation, so it localizes nothing. RelevanceToQuery passes an on-topic response that contradicts its source.