PRODUCTION SCENARIO
A screening model reports the same overall accuracy for two demographic groups. An analyst suspects that the model still misses far more of the true positive cases in one of the groups.
Which analysis will confirm this difference?
Answering here is anonymous. Nothing is saved unless you sign in.
Show answer and explanation
Answer: Compute the recall for each group and compare them.
Missing true positives is a recall failure, so comparing recall for each group is what surfaces it. Per-group accuracy is stated to be the same for both groups, so it will show nothing. The approval rate shows how many applicants the model passes, not how many it should have passed and missed. The share of training data from each group may explain why a gap exists but is not evidence that it does.