BIZZNA TUTOR statistics research

Previous students have recommended more explanation and clarification of the statistical tools used in this class and this appendix is a move in that direction. You will need an understanding of mean, standard deviation, the t-test, Fischer’s Exact Probability Test, and correlations. Let’s look at those, one at a time, and do some simple problems that should contribute to your mastery. This is certainly not a substitute for a statistics class, but is rather meant to give you a basic foundation so that you can intelligently handle the requirements of your pilot study.

Means and standard deviations

A mean is simply an average and calculating a mean is as simple as adding the scores and then dividing by the number of scores. When you work with your Likert items, this is done for you automatically. All you have to do is enter the data on the spreadsheet provided (yellow cells) and read the result. Just entering your raw data will not just give you the means, but the standard deviation will be calculated and provided for you as well. The median (score in the center of a distribution) and the mode (score appearing most frequently) will not be used in your research. These, along with the mean, are frequently grouped as measures or indicators of central tendency. We will use only the mean.

The standard deviation is a measure of variability. Are your scores homogenous, pretty tightly grouped such as samples 3, 4, and 5 in Table 1? If so, the standard deviation is a small value. Are your scores quite different, some strongly agree mixed with strongly disagree, for example (see samples 1 and 2 in Table 1)? If so, you’ll get a relatively large standard deviation. And if one group has a large SD and the other a small SD, then that’s food for thought and discussion. Specifically, why would one group have so much variation in their opinion? Why do they disagree among themselves? That would certainly be something worthy of investigation and discussion.

Let’s get some practice. Open the spreadsheet for doing t-tests and consider the raw data provided in this table.

Table 1. Raw data for statistics practice.

In your t-test spreadsheet, use the solid yellow block in either column C or column D. Enter the scores shown for sample 1 in Table 1 and copy the mean and SD in the spaces provided. Do the same for samples 2-5. For questions 6-8, assume a Likert scale of strongly agree = 5, agree = 4, neutral = 3, disagree = 2, and strongly disagree = 1.

Questions on means and standard deviations:

1. One sample shows no variability at all. Which one is it and why is that so?

2. Which two samples show the most variability in scores? What are their SD values?

3. Which two samples show the lowest amount of variation, excluding #3? What are their SD values?

4. By trial and error, list five scores that would be more variable than #2. What is the SD for your hypothetical sample?

5. By trial and error, list five scores that would be less variable than #4, but would show some variability. What is the SD for your sample?

6. Which sample mean indicates agreement with the survey item in question? Is it nearer ‘agree’ or ‘strongly agree’?

7. Which sample mean indicates disagreement with the survey item in question? Is it nearer ‘disagree’ or ‘strongly disagree’?

8. Which sample means indicate neutrality with respect to the question?

t-tests for independent samples

In your pilot study, you’ll compare two sets of sample scores by using the t-test spreadsheet. You will see means and standard deviations calculated for you following data entry and you’ll also see values for t that you can make a decision on regarding significance. With the total sample of ten subjects, a value of t equal to or greater than 2.31 will be required to meet the criterion for significant differences between the means. Look at the two columns in red font on the spreadsheet to see how the criterion value of t changes as the samples get larger.

Given that information, enter pairs of samples into the yellow cell blocks in your t-test spreadsheet after reading this whole paragraph. There’s no point in testing for differences of means between samples 1&2, 1&3, or 2&3. Why? Those means are all the same, 3.0, so there is no difference to test. In the real world, if you enter your pilot study data and it happens that the means are equal, the test for significance will be automatic and you’ll be provided with values of t to report. In such a case, t = 0…as far away from the criterion value of 2.31 as you can get. That makes sense, if the means are equal and you’re testing for significance of differences, doesn’t it? The opposite side of that coin is that the bigger the difference between means, the greater the probability that the two means are significantly different. The variability of scores, shown by the size of the SD is a factor, but we will ignore that at this point. Now you have five sets of means that you had calculated with the data from Table 1. Three of them are equal (3, 3, and 3) and then there are means of 1.6 and 4.2. Let’s do some testing of those pairs that are different. Go ahead now and enter pairs of data that you see in Table 2 into the two yellow columns of your spreadsheet. Then fill out Table 2. The last row will be absolute difference between means, no plus or minus signs (Just subtract the smaller mean from the larger). When you finish, check your results with the key provided at the end of this worksheet to make certain that you know how to use the t-test spreadsheet correctly.

Table 2. t-values and decisions on selected pairs of samples from Table 1.

Questions on the t-test results:

1. Once you finish filling out Table 2, you should find that the means of three of the seven pairs of samples are significantly different. For sets 4 & 5, the absolute difference between the means was 2.6 and common sense would lead you to believe those two means would be significantly different and they were. However, look at the difference between the means of the other pairs. They are all either 1.2 or 1.4, but in two cases we have significance and in four we do not. If the magnitude of the difference between means isn’t the only factor at work, explain to me what makes the difference in this table between significance and nonsignificance.

2. Look carefully at the two columns in red font on the spreadsheet for the answer to the three questions in this item. If you had six subjects per group, what is the criterion value for decision-making at alpha=.05? What would be the value required for ten subjects per group, a total of 20? What is the trend that you see in the relationship of sample size to the value of t required for significance?

3. You have already determined a t-value for 1 & 4. Now do the same for 4 & 1. In one case, you’ll enter the data for Group 1 in the left column and in the second case you’ll enter the data for Group 4 in the left column. The data for the two groups doesn’t change; only the order of entry is different. What is the only difference that you see in the value of t? Does that explain why we would not report a negative value of t and why the criterion value is always without sign, the absolute value? Please never report a t-value with a negative sign! It makes no sense at all.

Power (ability to avoid a Type 2 error –false positive – see p. 88)

Statistical power relates to the relative ability of detecting significant differences when the means are really different. Elsewhere in the Etext, a point was made that with the small sample sizes used in your pilot study, the probability of finding significance is slim or none. Here are a couple of exercises that will illustrate that point. You have already calculated t = 1.87 for Samples 1 and 4 data in Table 1. Now take that same data and enter it twice in the yellow blocks in your spreadsheet. For group one, enter sample #1 data of 5, 4, 3, 2, and 1 and continue down the column with the same numbers again 5,4,3,2,1. For group 2 on the spreadsheet, enter sample #2 date (see Table 1) twice as you did for sample #1. Be sure to change the values for the numbers in each group – see the two yellow cells to the right. You will then note that the means are identical (see the Table 1 values that you calculated), but the standard deviations are just a little lower. The big difference is in the value of t. Compare it with the criterion value for a sample of 20 subjects. You doubled the sample size and now find significant differences between the mean of 3.0 for sample 1 and the mean of 1.6 for sample 4. That will not always happen. This time we made sure that the additional scores were the same as those we started with. However, if you beef up your study with more subjects, there is no guarantee that the ones you add will think like the others! Random sampling will make that more likely to occur, but we deal with probabilities throughout statistical practice and there are no guarantees!

Questions on power

Add two more subjects to sample 1, scores that won’t affect the mean. What scores could that be? Let’s use 3 and 3! Now do the same for sample 4, adding scores of 1 and 2 to keep the mean fairly constant. Be sure to change the numbers in the two yellow spaces to the right…7 rather than 5 per group. Place those new numbers in cells F3 and F4.

1. What is the value of t?

2. Does this meet the criterion for significance?

3. What story does this tell about the importance of sample size?

Fischer Exact Probability Test for true/false and yes/no items

Most educational research is done at an alpha level of .05, meaning that if the probability calculated by the Fischer test is less than or equal to .05 we have a statistically significant difference. If the Fischer value exceeds .05 then the means are considered not to be significantly different. The size of the Fischer value is important. If you get one at .06 with your small sample, you’ll get excited and want to talk about it. On the other hand, if the value is .6, .8, or 1.0 it is a ho-hum-no difference situation.

Table 3. Hypothetical distributions for Fischer analysis.

Fisher problems. Enter these six sets of data in the yellow cells in the Fischer spreadsheet. Then copy the Fisher Exact Probability values into the proper cells with brown fill in this table. These are just examples of possible distributions and obviously there are more. As you can see, you have to have remarkable disagreement between samples to find significance. This, again, is a function of sample size and larger samples would increase the power, the probability of finding significance when the populations really are different. Look at the Exact Probability of Samples 3 and 4 and compare it with that of 5 and 6, both having exactly the same proportional distribution. This is another demonstration of the effect of sample size on statistical power.

Assignment on Fisher: Copy and paste the filled out Table 3 as your answers to the six problems.

Additional question: Look at the last two distributions on the left (samples 3 & 4, samples 5 &6). The proportions are the same for both. The only difference is that Samples 5 & 6 are exactly twice as large as sample 3 &4. You might want to try a distribution with samples three times as large as 3&4. You’d have the same proportional distribution in all three cases. What can you conclude regarding the importance of sample size in the Fisher Test?

Correlations

Please use the word correlation in this class only as a statistical term used to describe the degree of relationship. People not in the know often speak of things like the correlation between teachers’ and students’ thinking about something when they’re really meaning similarity. Don’t make that common mistake. Let’s look at some legitimate uses of the word correlation.

Suppose we want to describe the strength of the relationship between math ability as shown through final examination class data and math ability as shown in some sort of high stakes testing. How do the two relate? Open your correlations spreadsheet, please, and enter the data from this table. The teachers’ scores will be expressed as percentages for the year and the high stakes test score will be based on a maximum possible of 50. (We will express correlations in this form: r=.76)

Table 4. Hypothetical scores on two types of tests.

What is the correlation of these two sets of data? What do the results mean?

Values for correlations can range from 1.0 to -1.0 and are expressed in decimal form, often to two decimal places. If it is positive, it means that as values in one set of data increase, values in the other set increase as well. If the correlation is negative, it means that when one set increases, the other decreases. Drawing conclusions from correlations is sometimes done in error. You can never conclude anything in terms of cause and effect – just a relationship and what it might mean. Look at this next example.

Table 5. Incidence of shootings and parties over seven months.

What is the correlation between number of parties each month and shootings each month over this time period? What can we conclude? Do shootings cut down on parties? Does having fewer parties irritate people who then shoot more other folks?

There’s one more practical application of correlations that we will consider. In order to estimate the reliability of a Likert-type item, you could administer your survey to the same group on two occasions and calculate a correlation coefficient that shows the relationship between the two sets of scores. Our criterion for an acceptable survey item will be r>.7 (a correlation of at least .7). Ideally, an individual would give the same response each testing time, but life doesn’t work that way.

Table 6. Data from two administrations of Likert Item #2.

Calculate the correlation for this test-retest determination of the reliability of a Likert-type item. The same survey was administered to these subjects twice, one week apart.

What is the correlation coefficient? Is Likert Item #2 reliable? Explain.

In conclusion

Ideally, you’d have taken a statistics class, but realistically you can assume that you’ve mastered a pretty good collection of basic statistics concepts if you successfully answered those questions in red font. You’ll be able to handle the demands of this class, which include analyzing your data and then interpreting it so that you can do a good job of writing a results and discussion section for your paper.