PRODUCTION SCENARIO
A Generative AI Engineer must index 900 million 768-dimension vectors. The source table changes once per night, and the product owner has accepted up to 250 ms of added query latency in exchange for lower cost.
Which Databricks AI Search configuration fits these requirements?
Answering here is anonymous. Nothing is saved unless you sign in.
Show answer and explanation
Answer: A storage-optimized endpoint with a Delta Sync index in Triggered mode
Standard endpoints hold 320 million vectors at dimension 768, so 900 million needs a storage-optimized endpoint, which holds over one billion at that dimension and adds about 250 ms of query latency. Storage-optimized endpoints do not support Continuous sync, and a nightly source change only needs Triggered sync anyway. Splitting across three standard endpoints leaves the application to fan out and merge results itself, and a Direct Vector Access index still sits on a standard endpoint and makes the team update it by REST API.
Official Source: https://docs.databricks.com/aws/en/vector-search/vector-search ↗