Judging RAG answers that contradict the retrieved chunk

GenAI Engineer Application Development Medium Question 22 of 45

A Generative AI Engineer reviews a RAG assistant in MLflow. Traces show the retriever returning the correct policy chunk on 92% of questions, yet roughly one response in six states something that chunk contradicts.

Which built-in judge should the engineer score the application with?

Show answer and explanation

BRetrievalGroundedness, which checks the response against the retrieved chunks

RetrievalGroundedness assesses whether the response is factually supported by the provided context, which is the retriever-good, answer-wrong split the traces show, and it needs no ground truth. RetrievalSufficiency asks whether the chunks contain the answer, which the traces already establish, and it does require ground truth. Correctness also needs expected facts and blends retrieval with generation, so it localizes nothing. RelevanceToQuery passes an on-topic response that contradicts its source.

Source: docs.databricks.com/aws/en/mlflow3/genai/eval-monitor/conce…

Anonymous, no account needed. Your tally stays in this browser.

Next question Start 5 timed questions Browse all 45 original GenAI Engineer practice questions