Rater Errors and Rating Accuracy

Meta-analysis was used to determine the relationship between rater error measures and measures of rating accuracy. Data from 10 studies (N = 1,096) were used to estimate correlations between measures of halo, leniency, and range restriction and Cronbach's (1955) four measures of accuracy. The average correlation between error and accuracy was .05. No moderators of the error-accuracy relationship were found. Furthermore, the data are not consistent with the hypothesis that error measures are sometimes valid indicators of accuracy. The average value of the 90th percentile of the distribution of correlations (corrected for attenuation and range restriction) was .11. The use of rater error measures as indirect indicators of accuracy is not recommended.