PRODUCTION SCENARIO
A benefits assistant that is built on an Amazon Bedrock knowledge base returns weak answers. The company suspects the retriever returns the wrong passages, but it cannot tell whether the generation step also contributes to the problem.
Which evaluation will isolate the retrieval problem?
Answering here is anonymous. Nothing is saved unless you sign in.
Show answer and explanation
Answer: A retrieve-only RAG evaluation job
Amazon Bedrock RAG evaluations come in two types. A retrieve-only job bases its report on the data retrieved from the RAG source, so retrieval is judged on its own. A retrieve-and-generate job covers retrieval and generated summaries together, the mixture the company cannot separate. An automatic model evaluation job scores a model, not a knowledge base, and comparing generator models tests only generation.