Power for equivalence tests

Power is an important consideration for equivalence tests. When you think about power for equivalence tests, remember that the hypotheses for an equivalence test are different from the hypotheses for a typical test of population means.

Consider the difference between a 2-sample t-test and a 2-sample equivalence test. You use a 2-sample t-test to test whether the means of two populations are different. The hypotheses for the test are as follows:

·    Null hypothesis (H0): The means of the two populations are the same.

·    Alternative hypothesis (H1): The means of the two populations are different.

If the p-value for the test is less than alpha, then you reject the null hypothesis and conclude that the means are different.

In contrast, you use a 2-sample equivalence test to test whether the means of two populations are equivalent, within a range of values that you specify (the equivalence interval). The hypotheses for the test are as follows:

·    Null hypothesis (H0): The difference between the means is outside your equivalence interval. The means are not equivalent.

·    Alternative hypothesis (H1): The difference between the means is inside your equivalence interval. The means are equivalent.

If the p-value for the test is less than alpha, then you reject the null hypothesis and conclude that the means are equivalent.

Thus, power for an equivalence test is the likelihood that you will conclude that the difference of interest is within your equivalence limits, when this is in fact true. If your test has low power, you may mistakenly conclude that the difference is not within your equivalence limits when it actually is. The following factors affect the power of your test:

·    Sample size: Larger samples give your test more power.

·    Difference: When the difference is close to the center of the two equivalence limits, your test has more power.

·    Standard deviation: Lower variability gives your test more power.

·    Alpha: Higher values for alpha (a) give your test more power. However, a represents the probability of type 1 error. So increasing a increases your chance of claiming equivalence when it is not true.

More:

The following topics further describe power and other related topics:

What is power?

Prospective versus retrospective assessment of power

Hypothesis testing

Choosing an alpha level