PRODUCTION SCENARIO
A benefits assistant that is built on an Amazon Bedrock knowledge base returns weak answers. The team suspects that the retriever returns the wrong passages, but it cannot tell whether the generation step also contributes to the problem.
Which evaluation will isolate the retrieval problem?
Answering here is anonymous. Nothing is saved unless you sign in.
Show answer and explanation
Answer: A retrieve-only RAG evaluation job
Amazon Bedrock RAG evaluations come in two types. In a retrieve-only job the report is based on the data retrieved from the RAG source, so the team can judge retrieval quality on its own before the generated responses enter the picture.