PRODUCTION SCENARIO
A support product plans to send short routing requests to a smaller model and complex exception investigations to a stronger one, against a defined quality bar; the smaller lane exists because per-request cost at the routing volume rules out sending every request to the stronger model. An offline check shows the classifier that assigns each request to a lane is correct about 70% of the time.
What does that measurement imply for the routing design?
Answering here is anonymous. Nothing is saved unless you sign in.
Show answer and explanation
Answer: Raise the classifier's accuracy before splitting traffic across the models
Routing is recommended for tasks with distinct categories and only where classification can be handled accurately, so a classifier that misfiles three requests in ten sends real investigations to the weaker lane and breaks the quality bar before any saving is realised. Sending everything to the stronger model abandons the saving instead of fixing the cause, and the stem rules that out on per-request cost. Raising effort on the smaller model spends more tokens without giving it the capability the investigations need. And accepting the error rate trades away the stated quality bar, which is the constraint the design exists to hold.
Official Source: https://www.anthropic.com/engineering/building-effective-agents ↗