Many EEO analysts reference the federal Uniform Guidelines on Employee Selection Procedures (Guidelines) for direction: the group with the highest selection rate that also comprise a minimum of 2% of the applicant data may be defined as the Advantaged group. While this sounds intuitively appealing, what if this methodology results in misleading findings? This is concerning since the courts have accepted the Guidelines with deference and it carries so much weight in influencing EEO practice. Can it be possible for Guidelines methods to result in misleading findings?
Unfortunately, in this case, the answer is, "yes." The Guidelines' method for identifying the Advantaged group in individual race analyses may lead to false negative findings – a conclusion of no adverse impact, when there is in fact, adverse impact. A simple example will help to explain this not so uncommon phenomenon.
In the example below, 100 applicants competed for 36 positions. Applying Guidelines methods, NATs would be defined as the Advantaged group, because it meets the two prongs as defined in the Guidelines:
- NATs had the highest SR (Selection Rate) at 67%
- NATs applicants (3) comprised over 2% of all 100 applicants
ASN | BLK | HSP | NAT | WHT | |
---|---|---|---|---|---|
Hired | 3 | 2 | 3 | 2 | 26 |
Not Hired | 14 | 13 | 12 | 1 | 24 |
SR(%) | 18% | 13% | 20% | 67% | 52% |
FET (ADV=NAT) | 0.140 | 0.108 | 0.172 | 1.000 | 1.000 |
FET (ADV=WHT) | 0.022 | 0.009 | 0.039 | 1.000 | 1.000 |
ASN = Asian, BLK = Black, HSP = Hispanic, NAT = Native American, WHT = White |
The answer is fairly straightforward when one understands: 1) what is required to obtain statistically significant AI, and 2) the mechanics underlying the Guidelines method for identifying the Advantaged group:
- Statistical methods, such as the FET, need two (2) ingredients to obtain statistically significant AI:
- Effect Size: The size of the difference in SR between groups. In this example, the difference in SR between NAT (67%) and WHT (52%) applicants is small, and so the Effect Size is small. The difference in SR between NAT (67%) and BLK (13%) is large, and so the Effect Size for that comparison is large. Large Effect Size are more statistically powerful.
- Sample Size: The headcounts of the groups under consideration. In this example, there are only three (3) NAT applicants while there are 50 WHT applicants. Large Sample Size are more statistically powerful.
- Guidelines method for identifying the Advantaged group primarily rely on Effect Size (group with the highest SR), but largely ignores Sample Size (applicants need only comprise 2% of the total applicant pool).
Given this, what is the recommended method for proactive AI analysis when there are more than two groups?
Option 1 is to conduct a Power analysis to identify the Advantaged group. Power analyses are complex and complicated, and require significant technical background to conduct.
Option 2 is to simply analyze all possible combinations of SR comparisons for AI. While this may seem complicated, it is not, and the body of results are very helpful in proactive AI investigations.
In practice, Option 2 is recommended for almost all proactive AI investigations. Analyzing all combinations may seem laborious and complicated, but that effort will be rewarded with a very clear picture of the pattern of AI among the different subgroups, which is the goal of proactive AI investigations. As AI methods evolve and advance, it is important for EEO analysts to adapt accordingly. With greater access to more powerful computers and easier to use software, AI investigations need to be more comprehensive and exhaustive. This is the next step in AI investigations.