Evaluating the adverse impact ratio and its associated uncertainty: A Bayesian approach

When assessing adverse impact, the four-fifths rule (a measure of practical significance of the impact ratio) and ZD test (a statistical significance test of the difference in selection proportions) continue to be widely used in practice, despite disadvantages of using these two measures either in isolation or together in a disjointed manner. This study presents a novel approach that improves upon these problems by estimating a Bayesian impact ratio, which reflects a posterior probability distribution of the most probable values of the impact ratio. We examine the Bayesian impact ratio via a simulation that captures a range of selection scenarios with realistically varying parameters (e.g., total applicants, percentage of people from the protected class, and percentages of people hired from both subgroups). Our Bayesian priors follow a hypothetical court case by representing objective, weak, and strong plaintiff- and defendant-oriented assumptions of an adverse impact case. We demonstrate how to interpret the results of the Bayesian impact ratio, consider model sensitivity when evaluating different Bayesian priors, and make conclusions based on small samples, following the legal burden of proof (a preponderance of the evidence). Compared with the four-fifths rule and ZD test, we conclude that the Bayesian impact ratio reflects a more integrated and useful statistical approach when determining the presence of adverse impact with potential for clearer communication of results.

See the full article at Psychology, Public Policy, and Law