Topic drill
Retrieval-Augmented Generation (RAG)
Practice vector search, hybrid retrieval, chunking strategies, embeddings, and context relevance evaluation.
37 scenario questions from 6 tracks
Practice scenarios
- Letting a help desk assistant create and check support tickets
- Cutting vector storage cost for 400 million chunks in Bedrock
- Blocking RAG answers that add details not in the source
- What chunking does in Amazon Bedrock Knowledge Bases
- Evaluating knowledge base retrieval apart from the generator
- Cheapest vector store for 9 million pages with light traffic
- Stopping a Bedrock knowledge base from citing last year's policy
- Cheapest store for 60 million rarely searched embeddings
- Comparing two knowledge bases on retrieval alone
- Giving a manuals RAG assistant the steps around a match
- Answer passage outside the top 5 under a token budget
- Prompt cache lifetime with a query every two minutes
- Finding which agent step made a 31-second response slow
- Fixing chunks that split mid-sentence and lose section headings
- AI Search misses queries that quote exact part codes
- Judging RAG answers that contradict the retrieved chunk
- Calling a RAG chain registered in Unity Catalog over HTTPS
- Finding which of three changes dropped an evaluation score
- Parsing 20,000 scanned contracts in a Unity Catalog volume
- Chunks longer than the embedding model's 512-token limit
- Scoring production RAG traces for hallucination without labels
- High recall@50 but the LLM answers from the wrong chunk
- Repeated disclaimers crowding out content in retrieved chunks
- Making a custom Python step visible in an MLflow trace
- Isolating whether retrieval or generation failed in a RAG trace
- Guaranteeing verbatim quotes from 80-page regulatory PDFs
- RAG chunks missing issuer and quarter context in bank filings
- Hybrid search ranks the right passage below irrelevant ones
- Vector search missing exact part serial numbers in manuals
- Isolating 200 tenants in a shared vector database
- Fixed-window chunking splits steps from their warnings
- OpenAI vector store returns the wrong instrument generation
- Calculating recall at five for a 20-query retrieval test
- Seeing the chunks OpenAI file search actually retrieved
- Limiting OpenAI file search to one state's policy documents
- Retiring idle per-ticket OpenAI vector stores automatically
- Tuning vector store chunk size and overlap for long clauses