PRODUCTION SCENARIO
A hiring-screen model will be trained on 300,000 past applications. 91% of those applications come from one region. The team wants to know whether the data under-represents other regions before it spends resources on training.
Which check will answer this question at this stage?
Answering here is anonymous. Nothing is saved unless you sign in.
Show answer and explanation
Answer: Compute pre-training bias metrics such as class imbalance.
Pre-training data bias metrics are computed on the raw dataset before any model is trained, and class imbalance measures the difference in the number of members between facet values. Post-training metrics, drift monitoring, and feature attributions all require a trained model, so none of them can answer the question yet.