PRODUCTION SCENARIO
A travel company's generative AI assistant can call an internal refund API. Users have found that they can obtain larger refunds by telling the assistant to ignore its instructions.
Which change will most reliably stop this?
Answering here is anonymous. Nothing is saved unless you sign in.
Show answer and explanation
Answer: Enforce the refund limit in the API that the assistant calls.
The API is the only place that sees the actual amount and the caller's identity, so a limit enforced there holds whatever the user tells the assistant; AWS makes the same point about placing security controls outside agent code. An IAM role authorizes which API actions may be called, not the dollar value inside a request. A prompt attack filter scores wording probabilistically and is not an authorization boundary. Asking the user to confirm asks the attacker to confirm.