Power Analysis

One of the most common applications of statistics in the social and behavioral science is in testing null hypotheses. For example, a researcher wanting to compare two treatments will usually do so by testing the hypothesis that in the population there is no difference in the outcomes of these treatments. If this null hypothesis (H0) can be rejected, the researcher is likely to conclude that there is a real (i.e., non-zero) difference in treatment outcomes. The power of a statistical test is defined as the probability that a researcher will be able to reject a specific null hypothesis when it is in fact false. One of the key determinants of power is the degree to which the null hypothesis is false; if treatments have a very small effect, for example, it may be difficult to reject the hypothesis that they have no effect whatsoever. Effect size is not the only determinant of power, however. The power of a statistical test is a complex nonlinear function of the sensitivity of the test, the nature of the treatment effect, and the decision rules used to define statistical significance.