Answer passage outside the top 5 under a token budget

MLA-C02 Deployment and Orchestration of ML and AI Workflows Medium Question 29 of 30

A support chatbot calls the Retrieve operation of an Amazon Bedrock knowledge base and passes the top 5 results to a foundation model. Tests show that the passage that answers the question is usually in the top 25 results but often not in the top 5. Passing all 25 results to the model fixes accuracy, but it pushes input token cost and response latency past their targets.

Which solution will meet these requirements?

Show answer and explanation

ARetrieve 25 results, apply a Bedrock reranker model, and pass the top 5 reranked results

A reranker model meets the requirements. It scores how relevant each retrieved chunk is to the query and reorders the results, and it can be applied in the Retrieve and RetrieveAndGenerate operations. Retrieving 25 and keeping the 5 best after reranking sends fewer but more relevant passages, which AWS notes also lowers cost and latency. A larger context window still sends all 25 results, so cost and latency stay over target. Temperature changes how the model samples its output; the model still receives the same 5 passages. Whole-document results send much more text per result, which raises token cost.

Source: docs.aws.amazon.com/bedrock/latest/userguide/rerank.html

Anonymous, no account needed. Your tally stays in this browser.

Next question Start 5 timed questions Browse all 30 original MLA-C02 practice questions