Answer passage outside the top 5 under a token budget
MLA-C02
Deployment and Orchestration of ML and AI Workflows
Medium
Question 29 of 30
A support chatbot calls the Retrieve operation of an Amazon Bedrock knowledge base and passes the top 5 results to a foundation model. Tests show that the passage that answers the question is usually in the top 25 results but often not in the top 5. Passing all 25 results to the model fixes accuracy, but it pushes input token cost and response latency past their targets.
Which solution will meet these requirements?
Show answer and explanation
ARetrieve 25 results, apply a Bedrock reranker model, and pass the top 5 reranked results
A reranker model meets the requirements. It scores how relevant each retrieved chunk is to the query and reorders the results, and it can be applied in the Retrieve and RetrieveAndGenerate operations. Retrieving 25 and keeping the 5 best after reranking sends fewer but more relevant passages, which AWS notes also lowers cost and latency. A larger context window still sends all 25 results, so cost and latency stay over target. Temperature changes how the model samples its output; the model still receives the same 5 passages. Whole-document results send much more text per result, which raises token cost.
Source: docs.aws.amazon.com/
Anonymous, no account needed. Your tally stays in this browser.
Practice by topic: RAG questions Amazon Bedrock questions