PRODUCTION SCENARIO
A chat product running a model with adaptive thinking has a large system prompt, and the model now thinks before nearly every simple reply, adding latency. Cost is acceptable; latency is the concern.
What is the recommended first step?
Answering here is anonymous. Nothing is saved unless you sign in.
Show answer and explanation
Answer: Add prompt guidance to think when it will meaningfully improve the answer
Adaptive thinking's triggering is promptable, and the best practices say that when a model with a large system prompt thinks more often than you want, add guidance reserving thinking for multistep problems and answering directly otherwise. budget_tokens is the manual extended-thinking control and returns a 400 on Claude 4.7 and later. Capping max_tokens truncates the reply after the latency has already been spent. And raising effort makes Claude think on more requests and at greater length.