PRODUCTION SCENARIO
A Generative AI Engineer maintains a triage app that calls the OpenAI API directly with a key kept in the app's config. Leadership wants to trial Anthropic for the same task, apply per-team rate limits, and stop copying provider keys into every service.
Which change delivers all three goals?
Answering here is anonymous. Nothing is saved unless you sign in.
Show answer and explanation
Answer: Create an external model serving endpoint for each provider
External models in Model Serving expose providers such as OpenAI and Anthropic through one governed endpoint, keep provider keys in a single secure location, and support AI Gateway features including rate limiting and usage tracking. The app then swaps providers behind a consistent interface instead of embedding provider-specific code and keys.
Official Source: https://docs.databricks.com/aws/en/generative-ai/external-models/ ↗