When Your Top Choice Turns You Down. Effect of Rejected Offers on the Utility of Selection Tests

Estimates of the marginal utility, or productivity gains, associated with the use of valid selection tests are typically based on the average ability of those to whom jobs, positions, or treatments are initially offered, which can be estimated from the selection ratio. Because some offers are declined, and lower scoring candidates must be accepted in the place of higher scoring candidates who decline initial offers, the average ability of those actually selected will almost always be lower than the average ability of those who received the initial offers. As a consequence, the actual increase in utility associated with testing will generally be lower than it would be if all offers were accepted. Formulas are developed for calculating the average ability of those actually selected when the proportion of initial offers accepted is less than 100%. Three cases are considered: (a) those in which offers are declined at random, (b) those in which offers are declined by the highest scoring applicants, and (c) those in which test scores are related to the probability of accepting an offer. It is shown that under realistic circumstances, utility formulas currently used could overestimate utility gains by 30%-80%, and that under extreme circumstances, overestimation could be much worse. Further, such overestimation occurs even when tests are used only for screening rather than for rank-ordering candidates.