PRODUCTION SCENARIO
A Generative AI Engineer finds that a contractor's service principal consumed 60% of a shared pay-per-token endpoint's budget last month. The endpoint must stay available to internal teams at their current usage.
What should the engineer do?
Answering here is anonymous. Nothing is saved unless you sign in.
Show answer and explanation
Answer: Set a custom rate limit on the endpoint for that service principal
AI Gateway rate limits can be set as a per-user default and as custom limits for individual users or service principals, in queries or tokens per minute, so one caller is capped without affecting anyone else. Lowering the default, revoking access, or shrinking every request penalizes the internal teams the scenario protects.