PRODUCTION SCENARIO
A chat product running a model with adaptive thinking has a large system prompt, and the model now thinks before nearly every simple reply, adding latency. Cost is acceptable; latency is the concern.
What is the recommended first step?
Answering here is anonymous. Nothing is saved unless you sign in.
Show answer and explanation
Answer: Add prompt guidance to think only when it will meaningfully improve the answer
The best practices say adaptive thinking's triggering is promptable and, when the model thinks more often than desired with large system prompts, to add guidance such as reserving thinking for multistep problems and responding directly when in doubt. Higher effort elicits more thinking, and budget_tokens is not the lever on adaptive models.