PRODUCTION SCENARIO
A Generative AI Engineer's policy assistant cites pages that were retired last quarter but still sit in the wiki table feeding its Databricks AI Search index. Compliance requires that retired policies never appear in an answer.
What should the engineer do?
Answering here is anonymous. Nothing is saved unless you sign in.
Show answer and explanation
Answer: Filter the retired pages out of the source table so the index drops them
A Delta Sync index stays synced as its source is updated, so filtering retired pages out of the source table is what stops them reaching retrieval at all. The Hallucination guardrail denies responses that are not supported by their context, and an answer drawn from a retired page is fully supported by it, so the guardrail passes it. A prompt instruction is advisory and the retired text still enters the context, and a similarity threshold ranks by relevance, which says nothing about whether a policy is current.
Official Source: https://docs.databricks.com/aws/en/vector-search/vector-search ↗