PRODUCTION SCENARIO
A Generative AI Engineer sends claim summaries from a chat model on a Foundation Model APIs endpoint into a claims system that expects five fixed JSON keys. About 8% of responses arrive with extra prose or a missing key and fail to parse.
What should the engineer change in the request so the claims system can parse every response?
Answering here is anonymous. Nothing is saved unless you sign in.
Show answer and explanation
Answer: Set response_format to a json_schema type listing the five required keys
Structured outputs offer two response_format types, and only json_schema makes the output adhere to a declared schema; json_object asks for syntactically valid JSON of any shape, so a missing key still arrives and validating downstream just relabels the failure. Temperature 0 with a firmer instruction lowers the rate but leaves it stochastic. More max_tokens helps when the object was truncated, not when the model wrapped a complete object in prose.