Statistical Testing and Examples of Standardized Test in Research

Week 4 Guidance

Last week, we reviewed the normal distribution, or bell curve, for a continuous random variable. Under certain circumstances, we can use the normal distribution to approximate the binomial distribution. We also built the tools necessary for hypothesis testing. In particular, we discussed sampling from a population and how to use the normal distribution to sample means and standard deviations. You will review a variety of methods for statistical testing and will also locate an example of statistical testing in research.

Confidence Intervals If we collect a sample of data from a population the population of the sample is denoted by  and as we learned last week is an approximation for the population mean . The distribution of the sample means is approximately normal. So our sample mean may be near the center, or in rare cases on one of the two tails. We can use z-scores to cut the normal distribution into percentage bands on one side or two. For example, the area under a normal curve between +/- 1.96 standard deviations above or below the mean is approximately 0.95. Another interpretation is that only 5% of the distribution lies outside of the band between x-1.96 sigma and x+1.96 sigma. The five percent is split into two symmetric 2.5% pieces on either side. Put another way, we say that we are 95% confident that a sample drawn from a population will be within the bands. We call this our 95% confidence interval. It is a rare occurrence to be outside of this region (5% unlikely to be precise.)

In hypothesis testing, we establish a desired confidence level, denoted by alpha which gives us the percentage chance that our result is significant. At a 5% alpha, we are using a 95% confidence interval. We are seeking to establish that a sample mean is different from a population mean if it falls into the “exceptional” or rejection region, shaded in the picture above.

Check out this YouTube Video - https://youtu.be/m6sGjWz2CPg

It has many other videos after it is finished which can also help you with understanding Hypothesis testing, confidence levels and Z Tests. Just click on the right toggle. 

Hypothesis Testing – Population Mean with a known population standard deviation and a single sample We follow the steps from last week with the z-test. The z-statistic was also provided last week. The z-critical values can be obtained from a table or by using an inverse normal distribution calculator (In Excel for example). You just want to figure out if you have one or two tails and what the alpha value is. Then set the percentage to the left as the input to get the critical value as the output.

Population Mean with an unknown population standard deviation and a single sample with n>30

We use the single sample t-test. The observed statistic is similar to the z-statistic, but we use the sample standard deviation instead of the population standard deviation. For large degrees of freedom (n-1), the t-distribution looks very much like the normal distribution. The critical values are obtained in the same manner as the z-test.

Comparison of population means from two independent samples from the same population

A common application here is a treatment and control group from a population. There are really two cases here, one where we assume the two samples come from populations with equal variances and the other with equal variances not assumed. Often we use another test called the Levene’s test for equality of variances to know which one to use. The formulas are little more complicated but the basic idea is to pool the variances and then use that number for each of the two samples to compare their means using the sample distribution of the mean. We call this test the independent samples t-test for short.

Comparison of population means from two paired variables from the same population

The common application of this technique is the before and after. We take a group and give them an assessment of some type then administer some kind of intervention and administer the same assessment. This test helps us determine if the intervention led to a significant change in mean scores. The test works by taking the difference between the before and after (or two paired scores in general) and then running a single sample t-test on the differences against a presumed (null) difference of 0.

 

ANOVA

ANOVA stands for Analysis of Variance. You can think of this as an independent samples t-test for three or more groups. But that is really where the similarity ends. We construct the F statistic which is the ratio of the between group variance and the within group variance. Thus we analyze the ration of the variances. The F distribution is nothing like the normal or t-distributions. It is not symmetric and only has one tail. In addition, when we frame the hypothesis as a difference between the groups or not, the result is just that there is or is not a difference. In the case that we find a significant difference, then we have to use a post hoc analysis to determine where (between which two groups) the difference lies.

Nonparametric methods

Nonparametric methods are used when the normality or size condition is violated for any of the above tests or when the data are ordinal (rank ordering) in nature. The tests rely upon ranking the scores and applying an appropriate distribution. There are three main tests in this category – the Mann Whitney U test (independent samples t-test equivalent), the Wilcoxon (for paired samples), and the Kruskal-Wallis (ANOVA replacement).

Looking Ahead

In research, we often find that two variables show a relationship to each other. Whether or not that relationship is significant depends on the analysis around it. Next week, you will explore the concept of correlation and regression in research, and find examples of these concepts in research studies.

 

References:

Lind, D.A., Marchal, W. G., & Wathen, S.A. (2017). Statistical techniques in business and economics (17thed.).  Retrieved from http://connect.mheducation.com/class/