BIZZNA TUTOR statistics research

Directions for Unit 9

Data Analysis and Statistics

All the information you need for completing Unit 9 will be found on this directions sheet.

1. Keep the objective relevant to this unit in mind as you proceed: Following completion of this class the student will understand the basic differences among descriptive research, pre-experimental designs, true experimental designs, and quasi-experimental designs.

2. Complete the assigned readings.

3. You have three tasks to complete, administration of surveys, completion of a worksheet on statistics and a critical features analysis dealing with statistics.

a. Do not administer your surveys until you look at the edited version of your proposal. It will be returned to you, with possible changes to make, the day after you submit it to me. Wait until you look for possible problems before you administer your surveys. This is a critical point!

b. For the worksheet on pages 164-174 of your Etext, study the theoretical material carefully, follow the directions, and then answer the questions. Submit your answers. For each item, type the question in bold as a paragraph heading and then put your answer immediately following the question. There should be as many paragraphs as questions.

c. Critical Features Analysis – Statistics. Copy the table on the next page, fill in the blanks, and submit.

Critical Features Analysis on Statistics

(Based on Etext section on statistics following this table)

Situation	Best test
You want to predict IQ from GPA scores.
In Jones Elementary, math instruction in one 2nd grade class is through use of manipulatives, another 2nd grade class uses the new thought control technique, and a third 2nd grade class works under integrated yoga-pattern recognition theory.
How would you test mean achievement differences between Dr. Byrd’s two sections of Research?
Beginning-of-year class means on social studies concepts mastery are compared to end-of-year scores on the same variable
Means of seniors’ reading level scores at six high schools are to be compared. Assume that randomization has made the groups equivalent at the outset.
Is the distribution of two groups different on a true-false survey item?
Two matched classes, one having traditional math instruction and the other having a combination of traditional instruction with supplemental use of computer software aids…you want to compare the means of a posttest.
The same situation in the previous row, but this time you want to compare pretest scores to determine whether the two groups are equivalent before the experimental treatment is implemented.
What is the relationship, for those students who go to college, of high school GPA to family income?
You want to determine the relationship between math achievement and achievement in reading

STATISTICAL ANALYSIS

Objective: Following completion of this class, the student will be able to carry out some basic statistical testing, will have fundamental knowledge regarding selection of proper tests, and will understand the necessity for consulting statisticians prior to beginning a research project.

In the planning stages of any research you might do in the future, one of your first steps should be to consult an expert in statistics. Because you have no required statistics courses in your curriculum and because coverage in this class will be minimal, the probability is that you will need help. With a simple research design, you might be seeking simply confirmation of your plans for statistical analysis, but in a complex design or when group equivalence is in question, you might avoid serious difficulties later. Planning the project is really as important as execution of the study and not to include consideration of what eventual statistical treatment to use is folly. This expert that you consult can be a statistician in the math or psychology departments or someone in education who is up on the topic. All faculty members with doctorates have taken multiple classes in statistics, but like much of what is learned, what you don’t use, you lose. Your advisor can either help you with statistical planning or can point you to another faculty member for assistance. After graduation, you’ll find that many of your mentors will still be happy to assist you.

The primary challenge is in selecting the proper statistical test. You might use only descriptive statistics, the most important being means and standard deviations. What is the average score? Were scores fairly homogeneous or was there a widespread dispersion in one or both groups? You’ve probably dealt with these concepts in previous classes and these are important questions to address in descriptive studies. However, in your pilot studies you are looking for the effect of an independent variable (your two groups) on a dependent variable (the answer to a survey item). If you’re to test the difference between means of these dependent variables (survey items), how many groups are involved? If there are two as there will be in your pilot study, use a t-test. You could also use the t-test to determine whether significant differences between two groups exist at pretesting in an experimental design. Testing the differences between means of three or more groups entails using analysis of variance (ANOVA). In the latter case, if the groups are not equivalent at the outset, analysis of covariance (ANCOVA) can be run. Are you looking for relationships between variables? Use a correlation procedure (Relationship? – Correlation!). Are you looking to predict something from a set of scores? For example, graduate schools like to predict graduate performance (GPA) from GRE scores. Regression analysis will do the trick. When working with true/false or yes/no survey items like you will have in your pilot study, a procedure you can use is Fischer’s Exact Probability Test.

This might sound easy, but it isn’t really this simple. There’s more than one kind of t-test. You’ll be using a t-test for independent samples. If you test for differences between pretest and posttest of a single sample, you’d use a t-test for dependent samples. There is a whole set of what are called nonparametric tests which serve the same purpose as the t-tests and analysis of variance mentioned. There are different correlational procedures, depending in part on the kind of data you have. These are but examples of the kinds of things that the statistician will be interested in talking to you about and that will lead to the final decision on the approach to take. For your pilot study in this class, I will serve as your statistician and guide you through the proper statistical analysis procedures.

lllustration of complexity of selecting the proper nonparametric test

In executing your pilot study in this class you will work with mandated small sample sizes, just to go through the procedures while looking for Murphy’s Law to work. With small sample sizes, the power of the test is weak, as is discussed in the Etext section on sampling. A lack of power indicates that it is very difficult to detect significant differences between means. That isn’t a problem with the pilot study, but in a full-blown research project you’d like to have group numbers of 20 or more in some cases. You should place the following statement in the opening paragraph of your results and discussion section: While it is generally understood that statistically insignificant differences are not discussed, this pilot study had a mandated five subjects per group, almost assuring findings of no statistically significant differences. Therefore, there will be at least some discussion of findings from the statistical analysis of each survey item. (Copy and paste freely; there is no need to cite the source in this case.)

Fortunately, there are many easy-to-use software packages that can help you with the statistical treatment. Excel spreadsheets can have some calculation formulas built in and three of those will be provided for your use in this class. Computers have replaced older and more tedious calculation procedures and in doing so have lowered expectations of calculation errors. Statistics is no longer something to be feared. It is simply a tool to use in describing what you found and analyzing for significance. Remember, please seek expert assistance unless you have a formal background of several statistics courses. This particular course provides the bare minimum in statistical training. It should help you in planning and some understanding of what statistical treatment to choose, but it just might raise more questions than answers. If all you learned was that an expert’s advice is imperative in planning this part of your research, you’re ahead in this game!

You will have statistics assignments to complete prior to getting into your data analysis. You’ll learn how to use Excel to obtain means and standard deviations, to carry out t-tests, to do Fisher’s Exact Probability Test, and to use the Pearson-r correlation procedure for determining reliability of your Likert-type items. Finally, you’ll have an exercise that will illustrate the effect of sample size on power. This class will give you only a superficial exposure to using statistics in educational research. To really understand statistics, formal classwork in that field would be desirable.

Let’s look at the null hypothesis and make sure you understand what it means and how the statistical results relate to it. First, the null hypothesis is a statement that there will be no significant differences between the means. This is stated for a given alpha level, most often set at .05 in educational research. The alpha value is sometimes called the level of confidence or level of significance. Basically, what alpha = .05 means is that 5 times in 100 a sampling error will cause you to make a mistake by rejecting the null hypothesis when it is really true. Remember that in sampling we use random assignment to groups, but there is always the outside possibility that we end up with a biased sample. The silver lining to the 5% error is that 95% of the time when we reject the null hypothesis at alpha = .05 we are correct.

What we’ve just discussed is called a Type 1 error. There is another possible statistical mistake that we can make, actually that we most certainly will make in these pilot studies. It is a Type 2 error, failing to reject the null hypothesis when it is false. This happens often as a result of inadequate statistical power caused by a too-small sample. I know that dealing with these negatives and double negatives is difficult, but please be patient. When we make this Type 2 error, we do not reject the null hypothesis even when there is truly a difference between the two groups. What will occur in most of your analyses, with your very small samples, is that you will not reject your null hypotheses. You’ll look at the means of the two groups and they will appear to be really big differences, but your statistical testing says no the difference is not statistically significant. You know in your heart that the two groups’ perceptions are different, but statistics says otherwise. If you’re right, to remedy the situation in a full blown study, you would increase the power of your test by using a larger sample size and then would be able to reject the null and say with confidence that the means are really different. There is a truth table on the next page that lays out the possible outcomes of statistical testing.

Did you ever hear of a false negative pregnancy test? That’s when the test kit says that you aren’t pregnant, but you really are. You can get a false positive treadmill test. That is when the test shows you to have heart disease but you really don’t.

The bottom line in all of this is that you should attempt to avoid Type 2 errors by using large enough samples (20-30 per group as a rule of thumb). There will remain the possibility of making the Type 1 error from just accidently getting a nonrepresentative sample. There’s not a lot you can do about that beyond your doing your best in sampling. We live with that, the 5% of the time error when alpha is set at .05.

Table. Truth table for statistical testing outcomes.

	Fail to reject	Reject
When the null hypothesis is really true (means are not sig diff)	This is fine, as it should be.	This is a Type 1 error that is made 5% of the time (alpha). (You say the null is not true when it really is – false negative )
When the null hypothesis is really false (means are sig diff)	This is a Type 2 error (Beta) (You say the null is true when it really isn’t – false positive )	This is fine, as it should be.

Try to get a good feel for these very basic statistical concepts. As a beginning researcher, it is common to be insecure with statistics. Your professional life might lead you into taking one or more classes in statistics during which you will develop an in-depth understanding. Until that happens, please consult the experts regarding experimental design and statistics.

Suggested reading

Ferguson, G. A. (1981). Statistical analysis in psychology and education (5th ed.). New York:

McGraw-Hill.

Wike, E. L. (1971). Data analysis: A statistical primer for psychology students.

Chicago: Aldine-Atherton.