PRODUCTION SCENARIO
A hiring-screen model will be trained on 300,000 past applications, 91% of which come from one region. No model has been trained yet, and the team wants evidence about whether other regions are under-represented in the data.
Which check will answer this question at this stage?
Answering here is anonymous. Nothing is saved unless you sign in.
Show answer and explanation
Answer: Compute the class imbalance metric on the raw dataset.
Class imbalance measures the difference in the number of members between facet values and belongs to the data bias metrics computed on raw datasets before training; these are model-agnostic because they do not depend on model outputs. Recall difference needs predicted labels. Bias drift monitoring compares live inference data against a baseline and needs a deployed endpoint. SHAP attributions explain how a trained model weighed its inputs.