psy 635

Chapter 2

LOGICAL STEPS

OF CONDUCTING

QUANTITATIVE RESEARCH:

HYPOTHESIS-TESTING

PROCESS

LEARNING OBJECTIVES

� Understand the logic and purpose of the hypothesis-testing process in scientific research.

� Identify the components and application of alternative and null hypotheses.

� Examine the meaning of alpha level and commonly used criterion levels of alpha (α) used in research.

� Explore the elements used to choose an appropriate sta- tistic for use to test a null hypothesis.

c02 18 June 2012; 18:32:48

Martin, William E., and Krista D. Bridgmon. Quantitative and Statistical Research Methods : From Hypothesis to Results, John Wiley & Sons, Incorporated, 2012. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/ashford-ebooks/detail.action?docID=843632. Created from ashford-ebooks on 2018-10-11 18:20:20.

C op

yr ig

ht ©

2 01

2. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

.

� Understand decision rules associated with rejecting and failing to reject a null hypothesis.

� Realize the importance of including effect size and confi- dence interval information to further clarify making a decision concerning the null hypothesis.

The hypothesis-testing process is a logical sequence of steps toconduct the statistical analyses in a quantitative research study. Indeed,hypothesis testing is the most widely used statistical tool in scientific research (Salsburg, 2001, p. 114). However, we must remember that no method, including obtaining statistical results from hypothesis testing, is the absolute final answer to a research problem. As Snedecor and Cochran (1967) stated, “But the basic ideas in statistics assist us in thinking clearly about the problem, provide some guidance about the conditions that must be satisfied if sound inferences are to be made, and enable us to detect many inferences that have not good logical foundation” (p. 3).

HYPOTHESIS-TESTING PROCESS

There are six steps of the hypothesis-testing process that provide the procedure for conducting the statistical analyses used in this book. Descriptions and key con- cepts are discussed for each hypothesis step.

1. Establish the alternative (research) hypothesis (Ha). An alternative (research) hypothesis (Ha) is a speculative statement about

the relations between two or more variables used in a quantitative research study (Kerlinger & Lee, 2000). A researcher initially develops one or more research hypotheses about the direction and expected results of a study. In experimental and quasi-experimental research, the variables stated in an alternative hypothesis reflect the changes in an outcome (dependent variable) that can be attributed to a cause (independent variable) (Martin & Bridgmon,

c02 18 June 2012; 18:32:49

30 � CHAPT E R 2 Martin, William E., and Krista D. Bridgmon. Quantitative and Statistical Research Methods : From Hypothesis to Results, John Wiley & Sons, Incorporated, 2012. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/ashford-ebooks/detail.action?docID=843632. Created from ashford-ebooks on 2018-10-11 18:20:20.

C op

yr ig

ht ©

2 01

2. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

.

2009). Researchers focusing on the predictive relationships among variables often use the terms predictor variable (independent) and criterion variable (dependent).

An alternative hypothesis can be conceptualized as either a nondirectional hypothesis or a directional hypothesis. A researcher does not have a clear expectation about what direction the results of study will likely take when a nondirectional hypothesis is used. Typically, there is little or no previous evidence to inform the researcher as to what the results have been in the past represented by similar studies when a nondirectional alternative hypothesis is used. For example, if a researcher is studying the effects of cognitive- behavioral therapy (CBT) and interpersonal therapy (IPT) on weight gain among women with bulimia, a researcher might state a narrative nondirec- tional alternative hypothesis (Ha) as:

Ha: There will be significant differences in weight gain among women with bulimia when comparing the effects of cognitive-behavioral therapy to the effects of interpersonal therapy.

This nondirectional alternative hypothesis does not declare a direction as to whether the one therapy condition will produce more effects than the other therapy condition. We can also write this nondirectional hypothesis in the following symbolic format:

Ha : μ1 ðCBTÞ 6¼ μ2 ðIBTÞ

The symbol μ (mu) represents a population mean that we estimate using a sample mean (X ) when conducting inferential statistics to assess mean dif- ferences. In this case, we are hypothesizing that the population mean of weight gain resulting from CBT will be different from (not less than or greater than) the population mean of weight gain produced by IPT. We analyze sample means as estimates of the population means and might choose to use an independent t-test in this example.

A directional alternative hypothesis does state an expectation for the outcome of the study. Researchers usually design their studies using previous research to guide them. Since a major purpose of research is to integrate study results with related previous and ongoing studies and theory, directional hypotheses are commonly used by researchers.

c02 18 June 2012; 18:32:50

LOGICAL STEPS OF CONDUCTING QUANTITATIVE RESEARCH: HYPOTHESIS-TESTING PROCESS � 31 Martin, William E., and Krista D. Bridgmon. Quantitative and Statistical Research Methods : From Hypothesis to Results, John Wiley & Sons, Incorporated, 2012. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/ashford-ebooks/detail.action?docID=843632. Created from ashford-ebooks on 2018-10-11 18:20:20.

C op

yr ig

ht ©

2 01

2. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

.

The following directional alternative hypothesis could be written based on previous research studies showing cognitive-behavioral therapy to be more effective than interpersonal therapy in reducing weight gain among women with bulimia.

Ha: Cognitive-behavioral therapy will produce significantly higher weight gain among women with bulimia when compared to the effects of weight gain produced by interpersonal therapy.

This directional alternative hypothesis is a declaration that cognitive-behavioral therapy will produce more weight gain than interpersonal therapy among women with bulimia. Symbolically, the directional alternative hypothesis can be written as:

Ha : μ1 ðCBTÞ . μ2 ðIBTÞ

This symbolic alternative hypothesis states that the population mean of weight gain resulting from CBT will be larger (greater) than the effects of IPT on the population mean of weight gain.

2. Establish the null hypothesis (H0). This is the hypothesis that is tested statistically.

The null hypothesis (H0) can be viewed as: (1) the hypothesis whose nullification, statistically, would be taken as evidence in support of a specified alternative hypothesis, or (2) the hypothesis that there is “no difference between two sets of data with respect to some parameter, usually their means, or of no effect of an experimental manipulation on the dependent variable of interest” (Nickerson, 2000, p. 242); the latter is most commonly used in psychological research. The “no difference” approach is often referred to as a nil null hypothesis or a nil hypothesis.

We will be combining both approaches in the statistical nullification of the null hypothesis in support of a specified alternative hypothesis. Also, we will be testing no difference nulls and suspending judgment when we fail to reject nulls. A null hypothesis related to our previous alternative hypothesis example can be stated as:

H0: There will be no difference in weight gain among women with bulimia when comparing the effects of cognitive-behavioral therapy to the effects of interpersonal therapy.

c02 18 June 2012; 18:32:50

32 � CHAPT E R 2 Martin, William E., and Krista D. Bridgmon. Quantitative and Statistical Research Methods : From Hypothesis to Results, John Wiley & Sons, Incorporated, 2012. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/ashford-ebooks/detail.action?docID=843632. Created from ashford-ebooks on 2018-10-11 18:20:20.

C op

yr ig

ht ©

2 01

2. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

.

A “no difference in effects” approach is taken in testing the null hypothesis. Also, a statistical nullification of the null would provide evidence in support of the specified alternative hypothesis. The null can be stated symbolically as:

H0: μ1 ðCBTÞ ¼ μ2 ðIBTÞ The null hypothesis states that the population means of weight gain resulting from CBT and IPT will be equal. There will be no difference between the means resulting from either treatment condition.

When the null hypothesis is rejected, a significant difference is found between the sample means of weight gain (estimating the population means) produced by the treatment conditions. Moreover, we will conclude that this nullification provides evidence in support of the alternative hypothesis. If the sample means are not significantly different from each other, we have an inconclusive decision when we fail to reject the null hypothesis. We cannot say that there is absolutely no difference between the two means in this case. We instead suspend judgment as to the relation between the two means that are not significantly different from each other.

3. Decide on the risk that one is willing to take for being wrong if one rejects a true H0, thus making a Type I (alpha [α]) error. Also, make a decision about the risk one is willing to take for being wrong if one fails to reject a false H0, thus making a Type II (beta [β] error).

A researcher chooses (sets) an alpha level (α) prior to data analysis. Common alpha levels used in research are .001, .01, .05, and .10, with the two most common being α ¼ .05 and α ¼ .01. One of the founders of statistics, Sir Ronald A. Fisher, suggested using α ¼ .05 in the early 1900s (Fisher, 1925). His practice evolved into the four most used alpha levels ranging from .001 to .10. As Nickerson (2000) states, “If α is set at .05, say, and a significance test yields a value of p equal to or less than .05, the null hypothesis is rejected and the result is said to be statistically significant at that level” (p. 243).

We also need to consider the chances associated with making a beta error (Type II error) when we select an alpha level for a particular study. We do this by conducting an a priori power analysis, which tells us the probability in a proposed study of correctly rejecting the false null hypothesis in favor of an alternative hypothesis. The power analysis is conducted before a study begins

c02 18 June 2012; 18:32:50

LOGICAL STEPS OF CONDUCTING QUANTITATIVE RESEARCH: HYPOTHESIS-TESTING PROCESS � 33 Martin, William E., and Krista D. Bridgmon. Quantitative and Statistical Research Methods : From Hypothesis to Results, John Wiley & Sons, Incorporated, 2012. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/ashford-ebooks/detail.action?docID=843632. Created from ashford-ebooks on 2018-10-11 18:20:20.

C op

yr ig

ht ©

2 01

2. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

.

using an estimated alpha level, sample size, and effect size. Power analysis is discussed further in Chapter 3.

4. Decide on the appropriate statistic to be used to test the H0 and the associated sampling distribution to be used (e.g., z, t, F, r, R, χ2) on the assumption that H0 is true.

The next step in the hypothesis-testing process is to identify the appro- priate statistic and associated sampling distribution to use to test the null hypothesis. The sampling distribution is a theoretical probability distribution that provides the foundation for testing hypotheses statistically.

Several issues are examined to determine whether a certain statistic is appropriate to use for a particular analysis (testing of an H0) in a study. The choice of a statistic is based on elements related to: (1) the focus of the interplay among variables (e.g., relationships or differences), (2) the number of independent and dependent variables used in an analysis, (3) the scales of measurements of the dependent variables, (4) the number and relationships (dependent vs. independent) of participant groups being compared, and (5) the extent that underlying assumptions of the statistic are met. A statistical design process integrating these elements is presented in Chapter 4.

5. Draw a sample of size N; screen the data for accuracy, missing values, and outliers; and assess whether the underlying assumptions of the statistic being used are met.

It may be necessary to change a decision about the statistic chosen or to modify the data set based on your data screening results. Compute the sample statistic and compare the result to the critical value (CV) (e.g., z.95, t.99, F.999) of the sampling distribution. More commonly, a comparison is made between the exact probability value obtained from the computer-generated statistical result and the chosen alpha level to determine significance.

Initially in the data screening process, it is important to conduct data cleaning once the data is collected. Study data may be entered into a com- puterized statistical program by hand or imported or downloaded from another source. All methods of data compilation have the potential for varying degrees of imprecision. It is therefore important to use various methods to check the accuracy of data before conducting statistical analyses.

The extent and pattern of missing data are another focus of data screening. Missing data can occur from data handling error, from study

c02 18 June 2012; 18:32:50

34 � CHAPT E R 2 Martin, William E., and Krista D. Bridgmon. Quantitative and Statistical Research Methods : From Hypothesis to Results, John Wiley & Sons, Incorporated, 2012. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/ashford-ebooks/detail.action?docID=843632. Created from ashford-ebooks on 2018-10-11 18:20:20.

C op

yr ig

ht ©

2 01

2. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

.

participants failing to respond to items, or from participants discontinuing their involvement in a study. Missing data can make it difficult to interpret and generalize the results of statistical analyses to the population from which the study sample was drawn.

Data diagnostic procedures are undertaken after the data are determined to be accurate. Many statistical procedures, known as parametric statistics, require that certain underlying assumptions are met in order to use a given statistic with a data set. A parametric statistical process is undertaken when sample characteristics are used to estimate population parameters such as those from the known normal distribution, t-distribution, and F-distribution. For example, a one-way analysis of variance requires that the scores of the dependent variable approximate a normal distribution, their variances across groups are homogeneous or constant, and there is independence of observations. If the underlying assumptions are not found to be met, then a researcher may need to use a nonparametric statistic that does not require the same strict underlying assumptions. Another common alternative is to transform the data to minimize the effects of outliers, nonnormal distributions, heterogeneous variances, and dependence of observations.

Once the data are readied from data screening and diagnostics, the testing of the null hypothesis can be conducted using the statistical analysis. The statistical analysis significance probability (generated from the proba- bilities of the sampling distribution used) is compared to the selected alpha level. The researcher makes a decision to reject or fail to reject the null hypothesis.

6. Make a decision regarding the H0; either reject the H0 in favor of the Ha or fail to reject H0. Additionally, provide effects sizes and confidence intervals.

If the statistical significance probability is equal to or less than the alpha level, the null hypothesis is rejected. The decision of failing to reject the H0 is made if the statistical significance probability is greater than α. For example, in rejecting the null hypothesis when α ¼ .05, we will make a Type I error no more than five times in 100. There is only a 5 percent or less chance that we reject a true H0. This statistical decision making can be illustrated using the null hypothesis stated earlier.

c02 18 June 2012; 18:32:50

LOGICAL STEPS OF CONDUCTING QUANTITATIVE RESEARCH: HYPOTHESIS-TESTING PROCESS � 35 Martin, William E., and Krista D. Bridgmon. Quantitative and Statistical Research Methods : From Hypothesis to Results, John Wiley & Sons, Incorporated, 2012. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/ashford-ebooks/detail.action?docID=843632. Created from ashford-ebooks on 2018-10-11 18:20:20.

C op

yr ig

ht ©

2 01

2. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

.

(Narrative Null)

H0: There will be no difference in weight gain among women with bulimia when comparing the effects of cognitive-behavioral therapy to the effects of interpersonal therapy.

(Symbolic Null)

H0: μ1 ðCBTÞ ¼ μ2 ðIBTÞ

We will say that we found a statistical significance probability of p ¼ .003 and the alpha level of significance is α ¼ .05. The p ¼ .003 is less than the alpha criterion of α ¼ .05 so the rule is to reject the H0. We can conclude that there was a significant difference in weight gain resulting from a comparison of the CBT and IBT conditions. There is a less than 5 percent chance that we reject a true H0 ( p , .05). We expect that fewer than five times in 100 would there be no difference between the two means. There is another way to state this statistical decision of rejecting the null hypothesis by linking to sampling distribution theory. It could be said that if one repeatedly took two random samples from the same population and tested the differ- ences between their means, one would expect to get a difference that was significant at the .05 level about five times in 100 (Nickerson, 2000).

Now, let’s say that that we found a statistical significance probability of p ¼ .11 and the alpha level of significance is still α ¼ .05. The obtained p ¼ .11 is greater than the alpha criterion of α ¼ .05, so the rule is to fail to reject the H0 ( p . .05). We can conclude that it is inconclusive that CBT and IPT have a differing effect on weight gain among participants. When we fail to reject the null hypothesis, we cannot conclude that there is an absolute noneffect or no difference. We would not say that CBT and IPT were equal in their effects on weight gain. A nonsignificant result is an inconclusive result or indefinite decision.

Making a decision about the null hypothesis is not enough. Additional information is needed to help draw accurate conclusions about one’s study data. Cohen (1994) provided three recommendations to improve the null hypothesis significance testing (NHST) process. First, understand that there is no magic alternative to NHST. Second, before generalizing from your data,

c02 18 June 2012; 18:32:50

36 � CHAPT E R 2 Martin, William E., and Krista D. Bridgmon. Quantitative and Statistical Research Methods : From Hypothesis to Results, John Wiley & Sons, Incorporated, 2012. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/ashford-ebooks/detail.action?docID=843632. Created from ashford-ebooks on 2018-10-11 18:20:20.

C op

yr ig

ht ©

2 01

2. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

.

understand and improve the data using “detective work” methods such as data screening or exploratory data analysis (EDA). Third, routinely report effect sizes and report data in the form of confidence limits. Jones and Tukey (2000) proposed using confidence intervals when they are available and rec- ognizing the limitations with traditional NHST.

Jones and Tukey (2000) also proposed another conclusion-making process, emphasizing that a researcher should assess the sample data and entertain one of three conclusions: “(a) act as if μA � μB . 0; (b) act as if μA � μB , 0; or (c) act as if the sign of μA � μB is indefinite, that is, is not (yet) determined” (p. 412). Again, using our previous example, the first possible conclusion following statistical analysis is that μCBT � μIPT . 0, which reflects that CBT produces a higher weight gain than the IPT con- dition. The second possible conclusion is that μCBT � μIPT , 0 resulted in CBT producing less weight gain than the IPT condition. The final possible conclusion is that μCBT � μIPT is indefinite—that is, inconclusive.

It is important to remember that replication studies provide the tenability of scientific findings. Scientific conclusions result from a stream of well- designed studies in which data are compared across studies to formulate theory. There is no final scientific truth, but, instead, there is evolving scientific theory.

SUMMARY

The six steps of the hypothesis-testing process were discussed in this chapter. The hypothesis-testing process is the model or template that is used to learn the statistical analyses covered in the book. The hypothesis-testing process is the most widely used tool in science; however, there are limitations to be aware of.

PROBLEM ASSIGNMENT

The components of the hypothesis-testing process were presented in this chapter, including a rationale for decision making regarding the rejecting or failing to reject the null hypothesis. Examples are available on the companion website for you to practice your understanding of the hypothesis-testing process.

c02 18 June 2012; 18:32:51

LOGICAL STEPS OF CONDUCTING QUANTITATIVE RESEARCH: HYPOTHESIS-TESTING PROCESS � 37 Martin, William E., and Krista D. Bridgmon. Quantitative and Statistical Research Methods : From Hypothesis to Results, John Wiley & Sons, Incorporated, 2012. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/ashford-ebooks/detail.action?docID=843632. Created from ashford-ebooks on 2018-10-11 18:20:20.

C op

yr ig

ht ©

2 01

2. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

.

KEY TERMS

alpha level (α)

alternative (research) hypothesis (Ha)

a priori power analysis

beta error (Type II error)

confidence intervals

criterion variable

data screening process

dependent variable

directional hypothesis

effect sizes

independent variable

nil null hypothesis

nondirectional hypothesis

null hypothesis (H0)

nullification

parametric statistics

predictor variable

replication studies

sampling distribution

suspending judgment

μ (mu)

c02 18 June 2012; 18:32:51

38 � CHAPT E R 2 Martin, William E., and Krista D. Bridgmon. Quantitative and Statistical Research Methods : From Hypothesis to Results, John Wiley & Sons, Incorporated, 2012. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/ashford-ebooks/detail.action?docID=843632. Created from ashford-ebooks on 2018-10-11 18:20:20.

C op

yr ig

ht ©

2 01

2. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

.