Sample Size and Statistical Power

8.2 - Assignment: Sample Size and Power – Help file

Power: Note type II error occurs when the alternative hypothesis is true but for some reason we favor the null hypothesis. In other words, we don’t reject a false null hypothesis. The power of the hypothesis test is defined as the probability of not committing a type II error or the probability of rejecting a false null hypothesis. The power of a test of hypothesis ranges between 0 and 1. If the power is close to 1, you can conclude that the test is very good at detecting a false null hypothesis. On the other hand, if the power is close to zero then the test is not good at detecting false null hypothesis.

Power Analysis:

https://erau.instructure.com/courses/61765/files/11169037/download?wrap=1

https://www.youtube.com/watch?v=z-D6VmG7aLA

Effect size: The hypothesis test will tell us whether two means are significantly different or not. It will not tell us anything about the magnitude of difference between two groups. For this purpose, we calculate the effect size (review discussion on effect size related to t-tests we covered in week 5).

https://www.youtube.com/watch?v=wGlbyNBxEM8

Download G*Power Analysis Software (free) from the author’s website :

http://www.gpower.hhu.de/

Tutorials:

https://www.youtube.com/watch?v=nVBwhJ9gonQ

Example 1:

Assume you are comparing means of 4 groups using one-way ANOVA. Also assume you are expecting a medium effect size, level of significance α – 0.05 and a power of 0. 85. Estimate the sample size given these parameters and discuss how many data points are required for each group.

Open G*Power Analysis Software and choose Tests from the menubar, Means and Many Groups ANOVA:one-way (one independent variable)

Note: G*Power Effect size conventions: Small: 0.10 Medium: 0.25 Large: 0.40

The required sample size is 204 and this number needs to be distributed equally among the four groups (51 each).

Example 2:

Assume you are conducting a chi-square goodness of fit test and you are able to get a sample of only 55. Further assume you have the data in a 2 x 2 table. Calculate the power of the test given medium effect size.

Open G*Power Analysis Software and choose Tests from the menubar, Generic and Generic Chi-Square.

Note: G*Power Effect size conventions: Small: 0.10 Medium: 0.30 Large: 0.50. If the level of significance is not specified, assume it to be 0.05. Also note the df (degrees of freedom for a 2 x 2 table is (2-1)(2-1)= 1).

Note the calculated power is 0.521.

Additional discussion on power: (to respond to questions 3 and 4):

When you perform a t-test comparing two means with a relatively small sample size, there are two possible outcomes. 1) Reject the null hypothesis leading to the conclusion that the difference between the two means is statistically significant. In this case, if the power turns out to be small, it is not an issue as the difference between the groups have been established. 2) Do not reject the null hypothesis leading to the conclusion that there is no significant difference between the two means. In this case, if you have a low power you may not know whether there is actually no difference between the two groups, or your test is not able to detect the difference. Therefore, you need to increase the power and one way to do it is to increase the sample size. Level of significance and effect size are other factors that impact the power of a test. Review the link http://stattrek.com/hypothesis-test/power-of-test.aspx?Tutorial=AP for a discussion on the topic.