PRODUCTION SCENARIO
A product manager writes the success criterion for a new support-ticket classifier as "the model should classify tickets well." Engineering cannot tell whether a prompt change is an improvement.
How should the criterion be rewritten?
Answering here is anonymous. Nothing is saved unless you sign in.
Show answer and explanation
Answer: As a measurable target, such as an F1 score of at least 0.85 on a held-out set
Good criteria are specific and measurable, using quantitative metrics or well-defined scales; the docs contrast the vague "classify sentiments well" with an F1 target on a held-out test set. Without a number, there is no way to compare iterations.