Research Methods Journal :Method Comparison
7/30/20, 1)48 PMPrint
Page 1 of 33https://content.ashford.edu/print/Newman.2681.16.1?sections=navpo…t&clientToken=517d61d0-db98-8df2-52e3-a6d12160a350&np=navpoint-8
1.2 The Research Process With a broad understanding of the major research areas in psychology, we now turn our attention to the research process. How do psychologists conduct research? What are their goals? This section will answer these questions. This section will also compare quantitative and qualitative research, two different approaches to scientific inquiry.
The Scientific Method
What does it mean to draw conclusions based on science? Scientists across all quantitative disciplines use the same process of forming and testing their ideas. The overall goal of this research process—also known as the scientific method—is to draw conclusions based on empirical observations. In this section, we cover the four steps of the research process—hypothesize, operationalize, measure, and explain, abbreviated with the acronym HOME.
StepStep 1—Hypothesize 1—Hypothesize The first step in the research process turns an initial research question into a testable prediction, or hypothesis. A hypothesis is a specific statement about the relationship between two or more variables. For example, if we start with a question about the link between smoking and cancer, our hypothesis might be that smoking causes lung cancer. Or, if we want to know whether a new drug will be helpful in treating depression, we might hypothesize that drug X will lead to a reduction in depression symptoms. The next section of this chapter will cover hypotheses in more detail, but for now it is important to understand that the way a hypothesis is framed guides every other step of the research process.
StepStep 2—Operationalize 2—Operationalize Once a researcher develops a hypothesis, the next step is to decide how to test it. The process of operationalization involves choosing measurable variables to represent the elements of the hypothesis. In the depression-drug example, we need to decide how to measure both cause and effect; in this case we define the cause as the drug and the effect as reduced symptoms of depression. That is, what doses of the drug should we investigate? How many different doses should we compare? And, how will we measure depression symptoms? Will it work to have people complete a questionnaire? Or do we need to have a clinician interview participants before and after they take the drug?
An additional complication for psychology studies is that many of research questions deal with abstract concepts. Turning these concepts into measurable variables requires some art. For example, the abstract concept of happiness could be defined in countless different ways—being “happy” likely means something different to one individual than it does to his neighbors. To include happiness in a research study, we need to translate it into a more concrete concept, measured by a person’s score on a happiness scale or by the number of times a person smiles in a five-minute period, or perhaps even by a person’s subjective experience of happiness during an interview. Chapter 2 (2.2) will cover this process in more detail, with a discussion of
7/30/20, 1)48 PMPrint
Page 2 of 33https://content.ashford.edu/print/Newman.2681.16.1?sections=navpo…t&clientToken=517d61d0-db98-8df2-52e3-a6d12160a350&np=navpoint-8
guidelines for making these important decisions about the study.
StepStep 3—Measure 3—Measure Now that we have developed both our research question and our operational definitions, it is time to collect some data. The text will cover this process in great detail, dedicating Chapters 3 through 5 to the three primary approaches to data collection. Collection of data is a critical step in the research process, as researchers gather empirical observations that will help address their hypothesis. As Chapter 2 will explain, these observations can range from questionnaire responses to measures of brain activity, and they can be collected in a variety of ways, from online questionnaires to carefully controlled experiments. Regardless of the details of data collection, investigators will ultimately use these observations to make a decision.
StepStep 4—Explain 4—Explain After data have been collected, the final step is to analyze and interpret the results. The goal of this step is to return full circle to the initial research question and determine whether the results support the hypothesis. Recall the hypothesis that drug X should reduce depression symptoms. If we find at the end of the study that people who took drug X showed a 70% decrease in symptoms, this result would be consistent with the hypothesis. However, the explanation stage also involves thinking about alternative explanations and planning for future studies. What if depression symptoms dropped simply due to the passage of time? How could we address this concern in a future study? As it turns out, a fairly easy way of fixing this problem exists; Chapter 5 will cover that solution.
As Table 1.1 summarizes, the research process involves four stages: forming a hypothesis, deciding how to test it, collecting data, and interpreting the results. This process is used to draw conclusions across all scientific disciplines, regardless of whether research questions involve depression drugs, reading speed, or the speed of light in a vacuum.
Table 1.1 The HOME method
Stage of Process
Main Idea Example
HHypothesize Take a research question, turn it into a testable prediction
Question: Will my new drug help depression patients? Hypothesis: Drug X will reduce depression symptoms.
OOperationalize Turn the key concepts from your hypothesis into measurable variables
Depression can be measured using clinician interviews
MMeasure Choose and implement the best research design for your hypothesis
Compare two groups of people over time, half of whom have been given the new drug
EExplain Interpret your findings and make a decision about the state of your
If the people who take the new drug are less depressed at the end, that supports our
7/30/20, 1)48 PMPrint
Page 3 of 33https://content.ashford.edu/print/Newman.2681.16.1?sections=navpo…t&clientToken=517d61d0-db98-8df2-52e3-a6d12160a350&np=navpoint-8
hypothesis hypothesis
Research: Applying Concepts
Examples of the Research Process
To make the steps of the scientific method a bit more concrete, the following two examples show how they could be applied to specific research topics.
Example 1—Depression and Heart Disease
Depression affects approximately 20 million Americans, and 16% of the population will experience it at some time in their lives (NIMH, 2007). Depression is associated with a range of emotional and physical symptoms, including feelings of hopelessness and guilt, loss of appetite, sleep disturbance, and suicidal thoughts. This list has recently been expanded even further to include an increased risk of heart disease. Individuals who are otherwise healthy but suffering from depression are more likely to develop and to die from cardiovascular disease than those without depression. According to one study, patients who experience depression following a heart attack experience a fourfold increase in five-year mortality rates (research reviewed in Glassman et al., 2011).
Research Question
Based on these findings, we could ask the question, “Would it make sense to treat heart attack patients with antidepressant drugs?”
Recall that the goal of the scientific method is to take this research question, turn it into a testable hypothesis, and conduct a study that will test it. The following steps use the HOME method discussed earlier.
Step 1: Form a testable hypothesishypothesis from the research question.
We might predict that, “People who have had heart attacks and take prescribed antidepressants are more likely to survive in the years following the heart attack than those who do not take antidepressants.” We have taken a general idea about the benefits of a drug and stated it in a way that a research study can directly test.
Step 2: Decide how to operationalizeoperationalize the concepts in the study into measurable variables.
First, we would need to decide who qualifies as a “heart attack patient”: Would we include only those who had been hospitalized with severe heart attacks, or anyone with abnormal cardiac symptoms? These types of decisions will have implications for how we interpret the results.
7/30/20, 1)48 PMPrint
Page 4 of 33https://content.ashford.edu/print/Newman.2681.16.1?sections=navpo…t&clientToken=517d61d0-db98-8df2-52e3-a6d12160a350&np=navpoint-8
We would also need to decide on the doses of antidepressant drugs to use and the time period to measure survival rates. How long would we need to follow patients to obtain an accurate sense of mortality rates? In this case, earlier research had focused on five-year mortality rates, so that would be a reasonable time period for this study as well.
Step 3: MeasureMeasure the key concepts based on the decisions made in Step 2.
This step involves collecting data from participants and then conducting statistical analyses to test the hypothesis. We will cover the specifics of research designs beginning in Chapter 2 (2.1), but one good option would be to give antidepressant drugs to half of our sample and compare their survival rates with the half not given these drugs.
Step 4: ExplainExplain the results and tie the statistical analyses back into the hypothesis.
We would want to know whether antidepressant drugs did, indeed, benefit heart-attack patients and increase their odds of survival for five years. If so, our hypothesis is supported. If not, we would go back to the drawing board and try to determine whether a) something went wrong with the study, or b) antidepressant drugs actually do not have any benefits for this population. Answering these kinds of questions often involves conducting additional studies. Either way, the goal of this final step is to return to our research question and discuss the implications of antidepressant drug treatment for heart- attack patients.
Example 2—Language and Deception
In 1994, Susan Smith appeared on television claiming that her two young children had been kidnapped at gunpoint. Eventually, authorities discovered she had drowned her children in a lake and fabricated the kidnapping story to cover her actions. Before Smith was a suspect in the children’s deaths, she had told reporters, “My children wanted me. They needed me. And now I can’t help them” (The Washington Post, November 5, 1994, A15). Normally, relatives speak of a missing person in the present tense. The fact that Smith used the past tense in this context suggested to trained FBI agents that she already viewed them as dead (Adams, 1996).
Research Question
The story about Susan Smith highlights one way that people communicate differently when they are lying—they use past tense when present tense is more natural. This observation might lead us to ask, more broadly, “How do people communicate differently when they are lying versus when they are telling the truth?” We will again apply the HOME paradigm (or scientific method) to design a study that will ideally provide insight into this question.
Step 1: Form a testable hypothesishypothesis from the research question.
This example is somewhat more challenging because “communicating” can be defined in many ways. Thus, we need a hypothesis that will narrow the focus of our study. It turns out several studies have
7/30/20, 1)48 PMPrint
Page 5 of 33https://content.ashford.edu/print/Newman.2681.16.1?sections=navpo…t&clientToken=517d61d0-db98-8df2-52e3-a6d12160a350&np=navpoint-8
been conducted on the ways that people communicate when they are lying, ranging from variations in speech rate to differences in the use of certain types of words (for a review, see Depaulo et al., 2003). Based on one of these studies, we could offer the following specific prediction: “Liars communicate using more negative emotion (e.g., anger, fear) than truth-tellers do” (e.g., Newman, Pennebaker, Berry, & Richards, 2003). We have taken a general idea (“communicate differently”) and stated it in a way that can be directly tested in a research study (“use more negative emotion”).
Step 2: Decide how to operationalizeoperationalize the concepts in our study into measurable variables.
To determine measurable variables, we need to decide what counts as “using more negative emotion.” We could take the approach used in a previous study (Newman et al., 2003) and scan the words people use, looking for those reflecting emotions such as anger, anxiety, and fear. The theory behind this approach posits that the words people use reflect something about their underlying thought processes. In this case, people who are trying to lie will be more anxious and fearful as a result of the lie, and therefore use more words indicative of these negative emotions.
Step 3: MeasureMeasure the key concepts based on the decisions made in Step 2.
To measure the variables identified in Step 2, we must devise a way to determine whether and when people are lying. One way to do this in a research study is to instruct some people to lie and others to be truthful and then compare differences in the amount of negative emotion language between these groups.
Step 4: ExplainExplain the results and tie the statistical analyses back into the hypothesis.
We want to know whether people who were instructed to lie indeed used more words suggestive of negative emotion. If so, this outcome supports our hypothesis. If not, we would go back to the drawing board and try to determine whether a) the study design was flawed, or b) people in fact do not use more negative emotion when they lie. Either way, the goal of this final step is to return to our research question and discuss the implications for understanding language-based indicators of deception.
Goals of Science
In addition to sharing an overall approach to answering questions, all forms of scientific inquiry tend to adopt one of four overall goals. This section provides an overview of these goals, with a focus on how they apply to psychological research. We will encounter the first three goals throughout the course and use them to organize our discussion of different research methods.
DescriptionDescription One of the most basic research goals is to describe a phenomenon, including descriptions of behavior,
7/30/20, 1)48 PMPrint
Page 6 of 33https://content.ashford.edu/print/Newman.2681.16.1?sections=navpo…t&clientToken=517d61d0-db98-8df2-52e3-a6d12160a350&np=navpoint-8
Stockbyte/Thinkstock
Before a phenomenon can be explained it must first be described. For example, a survey might be used to collect information to describe the phenomenon of binge drinking.
attitudes, and emotions. Most people are probably very familiar with this type of research because it tends to crop up in everything from the nightly news to their favorite magazine. For example, if CNN reports that 60% of Americans approve of the president, it is describing a trend in public opinion. Descriptive research should always be the starting point when studying a new phenomenon. That is, before we start trying to explain why college students binge drink, we need to know how common the phenomenon is. We might, therefore, start with a simple survey that asks college students about their drinking behavior, and we might find that 29% of them show signs of dangerous binge drinking. Having described the phenomenon, we are in a better position to conduct more sophisticated research. (See Chapter 3 for more detail on descriptive research.)
PredictionPrediction A second goal of research is predicting a phenomenon. This goal takes us from describing the occurrence of binge drinking among college students to attempting to understand when and why they do it. Do students give in to peer pressure? Is drinking a way to deal with the stress of school? We could address these questions by using a more detailed survey that asked people to elaborate on the reasons that they drink. The goal of this approach is to understand the factors that make something more likely to occur. (See Chapter 4 for more detail on the process of designing surveys and conducting predictive research.)
ExplanationExplanation A third, and much more powerful, goal of research is to attempt to explain a phenomenon. This goal moves from predicting relationships to drawing stronger conclusions about causal links. Whereas predictive research attempts to find associations between two phenomena (e.g., college student drinking is more likely when students are stressed), explanatory research attempts to make causal statements about the phenomenon of interest (e.g., stress causes college students to drink more). This distinction may seem subtle at this point, but it is an important one, and closely related to the way that psychologists design their studies. (See Chapter 5 for more detail on explanatory research.)
ChangeChange The fourth and final goal of research is generally limited to psychology and other social-science fields: When we are dealing with questions about behaviors, attitudes, and emotions, we can sometimes conduct research to try to change the phenomenon of interest. Researchers who attempt to change behaviors, attitudes, or
7/30/20, 1)48 PMPrint
Page 7 of 33https://content.ashford.edu/print/Newman.2681.16.1?sections=navpo…t&clientToken=517d61d0-db98-8df2-52e3-a6d12160a350&np=navpoint-8
emotions are essentially applying research findings towards the goal of solving real-world problems.
In the 1970s, Elliot Aronson, a social psychologist at the University of Texas at Austin, was interested in ways to reduce prejudice in the classroom. Research conducted at the time was discovering that prejudice is often triggered by feelings of competition; in the classroom, students competed for the teacher’s attention. Aronson and his colleagues decided to change the classroom structure in a way that required students to cooperate in order to finish an assignment. Essentially, students worked in small groups, and each person mastered a piece of the material. Aronson found that using this technique, known as the “jigsaw classroom,” both enhanced learning and decreased prejudice among the students (Aronson, 1978). Read the details of Aronson’s study here: http://www.jigsaw.org/ (http://www.jigsaw.org/) .
Aronson’s research also illustrates the distinction between two categories of research. The first three goals we have discussed fall mainly under the category of basic research, in which the primary goal is to acquire knowledge, with less focus on how to apply the knowledge. Scientists conducting basic research might spend their time trying to describe and understand the causes of binge drinking but stop short of designing interventions to stop binge drinking. Researchers more often involve for this fourth goal of research in applied research, in which the primary goal is to solve a problem, with less focus on why the solution works. Scientists conducting applied research might spend their time trying to stop binge drinking without becoming caught up in the details of why these interventions are effective. Aronson’s research serves as a great example of how these two categories can work together. The basic research on sources of prejudice informed his applied research on ways to reduce prejudice, which in turn informed further basic research on why this technique is so effective.
One final note on changing behavior: Any time researchers set out with the goal of changing what people do, their values enter the picture. Inherent in Aronson’s research was the assumption that prejudice was a bad thing that needed to be changed. Although few people would disagree with him, he risked the difficulty of remaining objective throughout the research project. As we suggested earlier, the more emotionally involved we are in the research question, the more we have to be aware of the potential for bias, and the more closely we must pay attention to the data.
Approaches to Science: Quantitative versus Qualitative Research
Imagine for a moment that a psychologist wants to study depression across the life span. The researcher might approach this research question in one of two ways. She could design a survey that asked people to report their experiences with depression, as well as how often they had experienced various positive and negative life events. By conducting statistical analyses of these reports, she could gain a broad understanding of the relationships between life events and the development of depression. Alternatively, the investigator could spend her resources interviewing people who had been diagnosed with depression. Her goal is trying to understand what the experience felt like and whether people believed that it started in response to some major life event. This approach would provide a very deep understanding of the experience of depression from the inside out.
These alternative approaches highlight the differences between quantitative research and qualitative research,
7/30/20, 1)48 PMPrint
Page 8 of 33https://content.ashford.edu/print/Newman.2681.16.1?sections=navpo…t&clientToken=517d61d0-db98-8df2-52e3-a6d12160a350&np=navpoint-8
respectively. Quantitative research is a systematic and empirical approach that attempts to generalize results to other contexts. By surveying the population using structured scales, our hypothetical psychologist could learn about depression and life events in general. Qualitative research, in contrast, is a more descriptive approach that attempts to gain a deep understanding of particular cases and contexts. By interviewing depressed people in detail, the hypothetical psychologist could learn a great deal about how individuals experience depression.
The two approaches have traditionally been popular with different social science fields. For example, much of the current research in psychology is quantitative because the research aims for generalizable knowledge about behavior and mental processes. In contrast, much of the current research in sociology and political studies tends to be qualitative because research aims for a rich understanding of a particular context. To understand why college students around the country suffer from increased depression, quantitative methods are the better choice. To understand why the citizens of Egypt revolted against their government, then qualitative methods are more appropriate. However, many psychological phenomena are best understood by starting from the ground up, with a rich, qualitative understanding of people’s experiences. As later chapters will discuss, the qualitative approach has been used to gain insight into questions ranging from forming stigmatized identities to helping children cope with traumatic events.
In an ideal world, a true understanding of any phenomenon requires the use of both methods. That is, researchers can best understand depression if they both study statistical trends and conduct in-depth interviews with depressed people. Researchers can best understand binge drinking by conducting both surveys and focus groups. And investigators can best understand the experience of being bullied in school by both talking to the victims and collecting school-wide statistics. This text will discuss the ways that both approaches are used to shed light on pressing questions throughout the field of psychology. Table 1.2 compares the quantitative and qualitative approaches.
Table 1.2 Comparing quantitative and qualitative approaches
Quantitative Qualitative
Main Approach
Systematic, empirical, tries to generalize to other contexts
Descriptive, tries to gain rich understanding of a single context or example
Use of Not necessary; hypotheses sometimes the
7/30/20, 1)48 PMPrint
Page 9 of 33https://content.ashford.edu/print/Newman.2681.16.1?sections=navpo…t&clientToken=517d61d0-db98-8df2-52e3-a6d12160a350&np=navpoint-8
Hypotheses Starting point for all quantitative research result of qualitative study
Examples of Research
Study depression by surveying the population Study bullying by comparing reported incidents between schools
Study depression by interviewing patients Study bullying by interviewing bullies to understand their motivation
7/30/20, 1)48 PMPrint
Page 10 of 33https://content.ashford.edu/print/Newman.2681.16.1?sections=navp…t&clientToken=517d61d0-db98-8df2-52e3-a6d12160a350&np=navpoint-8
Getty Images/Handout
Nazi Lieutenant Colonel Adolf Eichmann’s claims during his trial that he was just “following orders”
1.3 Hypotheses and Theories The use of hypotheses is one of the key distinguishing features of quantitative research. Rather than making things up as they go along, scientists develop a hypothesis ahead of time and design a study to test this hypothesis. (Qualitative research, in contrast, often starts by gathering information and ends with a hypothesis for future inquiries.) This section covers the process of turning rough ideas about the world into testable hypotheses. We discuss the primary sources of hypotheses as well as several criteria for evaluating hypotheses. Watch the following video for an entertaining introduction to hypotheses and theories, which the chapter will then explore in detail: https://www.youtube.com/watch?v=lqk3TKuGNBA (https://www.youtube.com/watch?v=lqk3TKuGNBA) .
Sources of Research Ideas
Every study starts with an idea that researchers frame as a question. But where do all of these great ideas come from in the first place? Students are often nervous about starting a career in research for fear that they might not be able to come up with great ideas to test. In reality, though, ideas are easy to come by, a person knows where to look. The following material suggests some handy sources for developing research ideas.
Real-WorldReal-World Problems Problems A great deal of research in psychology and other social sciences is motivated by a desire to understand—or even solve—a problem in the world. This process involves asking a big question about some phenomenon and then trying to think of answers based on psychological mechanisms.
In 1961, Adolf Eichmann was on trial in Jerusalem for his role in orchestrating the Holocaust. Eichmann’s repeated statements that he was only “following orders” caught the attention of Stanley Milgram, a young social psychologist who had just earned a Ph.D. from Harvard University and who began to wonder about the limits of this phenomenon. To understand the power of obedience, Milgram designed a well-known series of experiments that asked participants to help with a study of “punishment and learning.” The protocol required them to deliver shocks to another participant—actually an accomplice of the experimenter—every time he got an answer wrong. Milgram discovered that two-thirds of participants would obey the experimenter’s commands to deliver dangerous levels of shocks, even after the victim of these shocks appeared to lose consciousness. These results revealed that all people have a frightening tendency to obey authority. We will return to this experiment in our discussion of ethics later in the chapter. Read more about Milgram and his landmark study on this website: http://www.-
7/30/20, 1)48 PMPrint
Page 11 of 33https://content.ashford.edu/print/Newman.2681.16.1?sections=navp…t&clientToken=517d61d0-db98-8df2-52e3-a6d12160a350&np=navpoint-8
throughout the Holocaust inspired Stanley Milgram to conduct a groundbreaking study about obedience to authority.
experiment-resources.com/stanley-milgram-experiment.html (http://www.experiment-resources.com/stanley-milgram-experiment.html) .
ReconciliationReconciliation and and Synthesis Synthesis Ideas can also spring from resolving conflicts between existing ideas. The process of resolving an apparent conflict involves both reconciliation, or finding common ground among the ideas, and synthesis, or merging all the pieces into a new explanation. In the late 1980s, psychologists Jennifer Crocker and Brenda Major noticed an apparent conflict in the prejudice literature. Based on everything then known about the development of self-esteem, members of racial and ethnic minority groups would have been expected to have lower-than-average self-esteem because of the prejudice they faced. However, study after study demonstrated that, in particular, African-American college students had equivalent or higher self-esteem than European- American students. Crocker and Major (1989) offered a new theory to resolve this conflict, suggesting that the existence of prejudice actually grants access to a number of “self-protective strategies.” For example, minority group members can blame prejudice when they receive negative feedback, making the feedback much less personal and therefore less damaging to self-esteem. The results of this synthesis were published in a 1989 review paper, which many people credit with launching an entire research area on the targets of prejudice.
LearningLearning From From Failure Failure Kevin Dunbar, a professor at Dartmouth University, has spent much of his career studying the research process. That is, he interviews scientists and sits in on lab meetings in order to document how people actually do research in the trenches. In a 2010 interview with Jonah Lehrer, Dunbar reported the shocking statistic that approximately 50 to 75% of research results are unexpected. Even though scientists plan their experiments carefully and use established techniques, the data are surprising more often than not. But even more surprising was the tendency of most researchers to discard the data if it did not fit their hypothesis. “These weren’t sloppy people,” Dunbar commented. “They were working in some of the finest labs in the world. But experiments rarely tell us what we think they’re going to tell us. That’s the dirty secret of science.” The trick, then, is knowing what to do with data that make a particular study seem like a failure (Lehrer, 2009).
According to Dunbar, the secret to turning failure into opportunity is twofold: First, question assumptions about why the study feels like a failure in the first place. Perhaps the data contradict the hypothesis but can be explained by a new one, or perhaps the data suggest a dramatic shift in perspective. Second, seek new and diverse perspectives to help in interpreting the results. Perhaps a cognitive psychologist can shed light on reactions to prejudice. Alternatively, perhaps an anthropologist knows what to make of the surprising results of a study on aggression. Some of the best and most fruitful research ideas have sprung from combining perspectives from different disciplines. Sometimes, all that a strange dataset needs is a fresh set of eyes.
Research: Thinking Critically
7/30/20, 1)48 PMPrint
Page 12 of 33https://content.ashford.edu/print/Newman.2681.16.1?sections=navp…t&clientToken=517d61d0-db98-8df2-52e3-a6d12160a350&np=navpoint-8
The Psychology Behind Pricing
Throughout this textbook, we will use short articles about research results as a way to illustrate key points in the text. Follow the link below to an article by William Poundstone, a bestselling author and expert on the psychology of pricing decisions. In this article, Poundstone discusses the peculiar appeal of prices ending in the number “9” and reviews recent research on this appeal by a pair of consumer psychology researchers. As you read the article, consider what you have learned so far about the research process, and then respond to the questions below.
https://www.psychologytoday.com/blog/priceless/201001/does-9-just-sound-cheap (https://www.psychologytoday.com/blog/priceless/201001/does-9-just-sound-cheap)
Think About It:
1. What hypothesis are Coulter and Coulter trying to test? Try to state this as succinctly as possible.
2. How was “perception of discounts” operationalized in their studies? 3. How were the key variables measured? 4. How do Coulter and Coulter explain their findings? Are there other possible alternative
explanations? 5. Are these studies primarily aimed at description, explanation, prediction, or change? Explain.
From Ideas to Hypotheses
Once a researcher develops a research question, the next step is to translate that question into a testable hypothesis—the first step in the HOME method. Broadly speaking, hypotheses are developed in one of two ways: bottom-up and top-down. This section explores these options in more detail.
Bottom-Up—FromBottom-Up—From Observation Observation to to Hypothesis Hypothesis Research hypotheses are often based on observations about the world around us. For example, people may have noticed the following tendencies as they observe those around them:
Teenagers do a lot of reckless things when their friends do them. Close friends and couples tend to dress alike. Everyone faces the front of the elevator. Church attendees sit and stand at the same time.
Based on this set of four observations, we could develop a general hypothesis about human behavior: People have a tendency to go along with the crowd and conform to group behaviors. This process of developing a general statement from a set of specific observations is called induction, and it is perhaps best understood as a
7/30/20, 1)48 PMPrint
Page 13 of 33https://content.ashford.edu/print/Newman.2681.16.1?sections=navp…t&clientToken=517d61d0-db98-8df2-52e3-a6d12160a350&np=navpoint-8
“bottom-up” approach. In this case, we have developed our hypothesis about conformity from the ground up, based on observing behavioral tendencies.
The process of induction is a very common and useful way to generate hypotheses. Most notably, this process serves as a great source of ideas that are based in real-world phenomena. Induction also helps us to think about the limits of an observed phenomenon. For example, we might observe the same set of conforming behaviors and speculate whether people will also conform in dangerous situations. What if smoke started pouring into a room and no one else reacted? Would people act on their survival instinct or conform to the group and stay put? Social psychologists Bibb Latané and John Darley (1969) conducted just such an experiment with groups of college undergraduates. Participants were asked to sit in a classroom and complete a survey. Meanwhile, the experimenters piped in smoke (actually dry ice) through the air vents. They hypothesized—and found—that the pressure to conform was stronger than the instinct to flee from a potential fire.
Top-Down—FromTop-Down—From Theory Theory to to Hypothesis Hypothesis The other approach to developing research hypotheses is to work down from a bigger idea. The term for these big ideas is a theory, which refers to a collection of ideas used to explain the connections among variables and phenomena. For example, the theory of evolution organizes knowledge about how species have developed and changed over time. One piece of this theory claims that human life originated in Africa and then spread to other parts of the planet. This idea in and of itself, however, is too big to test in a single study. Instead, researchers move from the “top down” and develop a specific hypothesis from a more general theory, a process known as deduction.
By developing hypotheses using a process of deduction, researchers’ biggest advantage is the ease of placing the study—and its results—in the larger context of related research. Because the hypotheses represent a specific test of a general theory, results can be combined with other research that tested the theory in different ways. For example, in the evolution example, a researcher might hypothesize that the fossils from human ancestors found in Africa would be older than those found in other parts of the world. If this hypothesis were supported, it would be consistent with the overall theory about human life originating in Africa. And as more and more researchers develop and test their own hypotheses about the origins of life, our cumulative knowledge about evolution continues to grow.
Table 1.3 presents a comparison of these two sources of research hypotheses, showcasing their relative advantages and disadvantages.
Table 1.3 Comparing sources of hypotheses
Deduction Induction
“Top-down,” from theory to hypothesis “Bottom-up,” from observation to hypothesis
Easy to interpret findings Can be hard to interpret without prior research
Helps science build and grow Helps understanding of the real world
7/30/20, 1)48 PMPrint
Page 14 of 33https://content.ashford.edu/print/Newman.2681.16.1?sections=navp…t&clientToken=517d61d0-db98-8df2-52e3-a6d12160a350&np=navpoint-8
oodelay/iStockphoto/Thinkstock
The theory of evolution is falsifiable, meaning that it could be disproved under the right conditions, such as the discovery of fossil evidence contradicting the theory.
Might miss out on new perspectives Great way to discover new ideas
Evaluating Theories
While experiments are designed to test one hypothesis at a time, the overall progress in a field is measured by the strength and success of its theories. If we think of hypotheses as individual combat missions on the battlefield, then theories are the overall battle plan. So, how do researchers know whether their theories are any good? Next, we cover four criteria that are useful in evaluating theories.
ExplainsExplains the the Past; Past; Predicts Predicts the the Future Future One of the most important requirements for a theory is that it be consistent with existing knowledge. If a physicist theorized that everything on earth should float off into space, that theory would conflict with millennia’s worth of evidence showing that gravity exists. Similarly, if a psychologist argued that people learn better through punishment than through rewards, that theory would conflict with several decades of research on learning and reinforcement. A new theory should offer a new perspective and a new way of thinking about familiar concepts, but it cannot be so creative that it clashes with what scientists already know. On a related note, a theory also has to lead to accurate predictions about the future, meaning that it has to stand up to empirical tests. There are usually multiple ways to explain existing knowledge, but not all of them will be supported as researchers test their assumptions in new circumstances. At the end of the day, the best theory is the one that best explains both past and future data.
TestableTestable and and Falsifiable Falsifiable Second, a theory needs to be stated in such a way that it leads to testable predictions. More specifically, a theory should be subject to a standard of falsifiability, meaning that the right set of conditions could prove it wrong (Popper, 1959). Calling something “falsifiable” does not mean it is false, only that if it were false, demonstrating its falsehood would be possible. The Darwinian theory of evolution offers an example of this criterion. One of the primary components of evolutionary theory is the idea that species change and evolve from common ancestors over time in response to changing conditions. So far, all evidence from the fossil record has supported this theory —older variants of species always appear farther down in a fossil layer. If conflicting evidence ever were to appear, however, it would deal a serious blow to the theory. The biologist J. B. S. Haldane was once asked what kind of evidence could possibly disprove the theory of natural selection, to which he replied, “fossil rabbits in the Pre-Cambrian era”—that is, a modern version of a
7/30/20, 1)48 PMPrint
Page 15 of 33https://content.ashford.edu/print/Newman.2681.16.1?sections=navp…t&clientToken=517d61d0-db98-8df2-52e3-a6d12160a350&np=navpoint-8
mammal buried in a much older fossil layer (Ridley, 2004).
Research: Thinking Critically
Intelligence, Politics, and Religion
Follow the link below to an article by Daniela Perdomo, a staff writer and editor for Alternet. In this article, Perdomo reviews the controversy over a recent scientific study claiming that liberals and atheists are more intelligent. As you read the article, consider what you have learned so far about the research process, and then respond to the questions below.
http://www.alternet.org/story/145903/controversy_grows_over_study_claiming_liberals _and_atheists_are_smarter (http://www.alternet.org/story/145903/controversy_grows_over_study_claiming_liberals_and_atheists_are_smarter)
Think About It:
1. What general theory is Kanazawa trying to test? How does the theory differ from his specific hypothesis?
2. How did Kanazawa operationalize liberalism and intelligence in his research? Are there problems with the way these constructs were operationalized? Explain.
3. What were Kanazawa’s main findings? How is the strength of this evidence influenced by his research methods?
4. Why do you think this research is controversial? If Kanazawa’s methodology were more rigorous, would it still be controversial?
ParsimoniousParsimonious Third, a theory should strive to be parsimonious, or as simple and concise as possible without sacrificing completeness. (Or, as Einstein [1934] famously quipped during a lecture at Oxford: “Everything should be made as simple as possible, but no simpler” [p. 165].) One helpful way to think about this criterion is in terms of efficiency. Theories need to spell out the components in a way that represents everything important but does not add so much detail that they become hard to understand. This means that theories can lack parsimony either because they are too complicated or because they are too simple.
At one end of this spectrum, Figure 1.1 presents a theoretical model of the causes of malnutrition (Cheah et al., n.d.). This theory does a superb job of summarizing all of the predictors of child malnutrition across multiple levels of analysis. The theory’s potential problem, though, is that it becomes too complicated to test.
7/30/20, 1)48 PMPrint
Page 16 of 33https://content.ashford.edu/print/Newman.2681.16.1?sections=navp…t&clientToken=517d61d0-db98-8df2-52e3-a6d12160a350&np=navpoint-8
Figure 1.1: Predictors of malnutrition
Figure 1.1 presents a theoretical model of the causes of malnutrition.
At the other end of the spectrum, Figure 1.2 shows the overall theoretical perspective behind behaviorism. In the early part of the 20th century, the behaviorist school of psychology argued that everything organisms do could be represented in behavioral terms, without any need to invoke the concept of a “mind.” The overarching theory looked something like Figure 1.2, with the “black box” in the middle representing mental processes. Nevertheless, the cognitive revolution of the 1960s eventually displaced this theory, as it became clear that behaviorism was too simple. To strike an ideal balance, then, a researcher constructs a theory in a way that includes only the necessary pieces, nothing unnecessary.
Figure 1.2: The behaviorist model
Figure 1.2 presents the overall theoretical perspective behind behaviorism. The “black box” in the middle represents mental processes.
7/30/20, 1)48 PMPrint
Page 17 of 33https://content.ashford.edu/print/Newman.2681.16.1?sections=navp…t&clientToken=517d61d0-db98-8df2-52e3-a6d12160a350&np=navpoint-8
Figure 1.3: The cycle of science
PromotesPromotes Research Research Finally, science is a cumulative field, which means that a theory is really only as good as the research it generates. To state it more bluntly: A theory is essentially useless if no one follows up on it with more research. Thus, one of the best bases for evaluating a theory is whether it encourages new hypotheses. Consider the following example, drawn from real research in social psychology. Since the early 1980s, Bill Swann and his colleagues have argued that people prefer consistent feedback to positive feedback, meaning that they would rather hear things that confirm what they think of themselves. One provocative hypothesis arising from this theory proposes that people with low self-esteem are more comfortable with a romantic partner who thinks less of them than with one who thinks well of them. This hypothesis has been tested and supported many times in a variety of contexts and continues to draw people in because it offers a compelling explanation for why some people stay in bad relationships—a phenomenon that is regrettably recognizable. (For a review of this research, see Swann, Rentfrow, & Guinn, 2005.)
The Cycle of Science
Now, let us take a step back and look at the big picture. We have covered the processes of developing and evaluating both broad theories and specific hypotheses. Of course, none of these pieces occurs in isolation; science is an ongoing process of updating and revising our views based on what the data show. This overall process of quantitative research works something like the cycle depicted in Figure 1.3. Researchers start with either an overall theory or a set of observations about how concepts relate to one another and use this to generate specific, testable, and falsifiable hypotheses. These hypotheses then form the basis for research studies, which generate empirical data. Based on these data, we may have reason to suspect the overall theory needs to be refined or revised. And, so, we develop a new hypothesis, collect some new data, and either confirm or do not confirm our suspicion. The process does not end there, however: other researchers may see a new perspective on our theory and develop their own hypotheses, which lead to their own data and possibly to a revision of the theory. The scientific approach may strike some as a slow and strange approach to problem solving, but it is the most objective one available.
Consider an example of how this cycle works in real life. In the 1960s, social psychologists were beginning to
7/30/20, 1)48 PMPrint
Page 18 of 33https://content.ashford.edu/print/Newman.2681.16.1?sections=navp…t&clientToken=517d61d0-db98-8df2-52e3-a6d12160a350&np=navpoint-8
study the ways that people explain the behavior of others (e.g., when someone cuts me off in traffic, I tend to assume he is a jerk.) One early theory, called “correspondent inference theory,” argued that people would come up with these explanations in a rational way. For example, if we read a persuasive essay but then learn that the author was assigned a position on the topic, we should refrain from drawing any conclusions about the writer’s actual position. However, research findings demonstrated just the opposite. In a landmark 1967 study, participants actually ignored information about whether authors had chosen their own position on the issue, assuming instead that whatever they wrote reflected their true opinions (Jones & Harris, 1967). In response to these data (and similar findings from other studies), the correspondent inference theory was gradually revised to incorporate what was termed the “fundamental attribution error”—people tend to ignore situational influence and assume that all behavior simply reflects the person’s own disposition. The study’s authors developed a theory, came up with a specific hypothesis, and collected some empirical data to test it. But because the data ran counter to the theory, the theory was ultimately revised to account for the empirical evidence. In this particular case, the cycle of research on understanding the fundamental attribution continues to this day, over 50 years later.
Proof and Disproof
While on the subject of adjusting theories, think about the notions of “proof” and “disproof.” Because science is a cumulative field, decisions about the validity of a theory are ultimately made based on results of several studies from several research laboratories. This means that a single research study has rather limited implications for an overall theory. This also means that a researcher must use the concepts of proof and disproof in the correct way. We will elaborate on this as we move through the course, but for now we can rely on two very simple rules:
1. If the data from one study are consistent with our hypothesis, we support the hypothesis rather than “prove” it. In fact, research almost never proves a theory, but statistical tests can at least suggest how confident to be in our support.
2. If the data from one study are not consistent with our hypothesis, we fail to support the hypothesis. As the course will discuss, many factors can cause a study to fail; these are often a result of flaws in the design rather than flaws in the overall theory.
7/30/20, 1)48 PMPrint
Page 19 of 33https://content.ashford.edu/print/Newman.2681.16.1?sections=navp…t&clientToken=517d61d0-db98-8df2-52e3-a6d12160a350&np=navpoint-8
2.1 Overview of Research Designs As Chapter 1 explained, scientists can have a wide range of goals when they begin a research project, everything from describing a phenomenon to changing people’s behavior. It turns out that these goals will dictate different approaches to answering a research question. That is, researchers will approach the problem of describing voting patterns differently than they would approach the problem of how to increase voter turnout. These approaches are called research designs, or the specific methods that are used to collect, analyze, and interpret data. The choice of a design is not one to be made lightly; the way an investigator collects data trickles down to decisions about how to analyze the data and about the kinds of conclusions that can be drawn from the results. This section provides a brief introduction to the three main types of design— descriptive, correlational, and experimental.
Descriptive Research
Recall from Chapter 1 that a research study can have the basic goal of describing a phenomenon. If a research question centers around description, then the research design falls under the category of descriptive research, in which the primary goal is to describe thoughts, feelings, or behaviors. Descriptive research provides a static picture of what people are thinking, feeling, and doing at a given moment in time, as the following examples of research questions illustrate:
What percentage of doctors prefer Xanax for the treatment of anxiety? (thoughts) What percentage of registered Republicans vote for independent candidates? (behaviors) What percentage of Americans blame the president for the economic crisis? (thoughts) What percentage of college students experience clinical depression? (feelings) What is the difference in crime rates between Beverly Hills and Detroit? (behaviors)
What these five questions have in common is an attempt to get a broad understanding of a phenomenon without trying to delve into its causes.
The crime-rate example highlights the main advantages and disadvantages of descriptive designs. On the plus side, descriptive research is a good way to achieve a broad overview of a phenomenon and may inspire future research. It is also a good way to study things that are difficult to translate into a controlled experimental setting. For example, crime rates can affect every aspect of people’s lives, and this importance would likely be lost in an experiment that staged a mock crime in a laboratory. On the downside, descriptive research provides a static overview of a phenomenon and cannot explore the reasons for it. A descriptive design might tell us that Beverly Hills residents are half as likely as Detroit residents to be assault victims, but it would not reveal the underlying reasons for this discrepancy. (If we wanted to understand why this was true, we would use one of the other designs.)
Descriptive research can be either qualitative or quantitative; in fact, the large majority of qualitative research falls under the category of descriptive designs. Descriptions are quantitative when they attempt to make comparisons or to present a random sampling of people’s opinions. The majority of our example questions above would fall into this group because they quantify opinions from samples of households, or cities, or
7/30/20, 1)48 PMPrint
Page 20 of 33https://content.ashford.edu/print/Newman.2681.16.1?sections=navp…t&clientToken=517d61d0-db98-8df2-52e3-a6d12160a350&np=navpoint-8
Johnathon Henninger/Connecticut Post/AP Images
Dr. Oliver Sacks studied how people with neurological damage formed and retained memories.
college students. Good examples of quantitative description appear in the “snapshot” feature on the front page of USA Today. The graphics represent poll results from various sources; the snapshot for May 15, 2015, reported that 90% of Americans crave more “variety” in their home-cooked meals (i.e., thoughts). View a current gallery of these snapshot graphs here: http://www.usatoday.com /services/snapshots/gallery/ (http://www.usatoday.com/services/snapshots/gallery/)
Descriptive designs are qualitative when they attempt to provide a rich description of a particular set of circumstances. A powerful example of this approach can be found in the work of the late neurologist Oliver Sacks. Sacks wrote several books exploring the ways that people with neurological damage or deficits are able to navigate the world around them. In one selection from The Man Who Mistook His Wife for a Hat, Sacks (1998) relates the story of a man he calls William Thompson. As a result of chronic alcohol abuse, Thompson developed Korsakov’s syndrome, a brain disease marked by profound memory loss. The memory loss was so severe that Thompson had effectively “erased” himself and could remember only scattered fragments of his past.
Whenever Thompson encountered people, he would frantically try to determine who he was. He would develop hypotheses and test them, as in this excerpt from one of Sacks’s visits:
I am a grocer, and you’re my customer, right? Well, will that be paper or plastic? No, wait, why are you wearing that white coat? You must be Hymie, the kosher butcher. Yep. That’s it. But why are there no bloodstains on your coat? (p. 112)
Sacks concluded that Thompson was “continually creating a world and self, to replace what was continually being forgotten and lost” (p. 113). With this story, Sacks helps illuminate Thompson’s experience and fosters readers’ gratitude for the ability to form and retain memories. This story also illustrates the trade-off in these sorts of descriptive case studies: Despite all its richness, we cannot generalize these details to other cases of brain damage; we would need to study and describe each patient individually.
Correlational Research
Recall from Chapter 1 that research studies can also have the goal of trying to predict a phenomenon. If a research question centers around prediction, then the research design falls under the category of correlational research, in which the primary goal is to understand the relationships among various thoughts, feelings, and behaviors. Examples of correlational research questions include:
Are people more aggressive on hot days?
7/30/20, 1)48 PMPrint
Page 21 of 33https://content.ashford.edu/print/Newman.2681.16.1?sections=navp…t&clientToken=517d61d0-db98-8df2-52e3-a6d12160a350&np=navpoint-8
Figure 2.1: Correlation is not causation
Are people more likely to smoke when they are drinking? Is income level associated with happiness? What is the best predictor of success in college? Does television viewing relate to hours of exercise?
What these questions have in common is the goal of predicting one variable based on another. If we know the temperature, can we predict aggression? If we know a person’s income, can we predict her level of happiness? If we know a student’s SAT scores, can we predict his college GPA?
These predictive relationships can turn out in one of three ways (Chapter 4 will provide more detail about each): A positive correlation means that higher values of one variable predict higher values of the other variable. For instance, more money is associated with higher levels of happiness, and less money is associated with lower levels of happiness. The key is that these variables move up and down together, as the first row of Table 2.1 shows. A negative correlation means that higher values of one variable predict lower values of the other variable. For example, more television viewing is associated with fewer hours of exercise, and fewer hours of television is associated with more hours of exercise. The key is that one variable increases while the other decreases, as the second row of Table 2.1 illustrates. Finally, worth noting is a third possibility, which is no correlation between two variables, meaning that we cannot predict one variable based on another. In brief, changes in one variable are not associated with changes in the other, as seen in the third row of Table 2.1.
Table 2.1: Three possibilities for correlational research
Outcome Description Visual
Positive Correlation
Variables go up and down together. For example: Taller people have bigger feet, and shorter people have smaller feet.
Negative Correlation
One variable goes up, and the other goes down. For example: As a driver’s speed goes up, the time it takes to finish the trip decreases.
No Correlation
The variables have nothing to do with one another. For example: Shoe size and number of siblings are completely unrelated.
Correlational designs are about testing predictions, but we are still unable to make causal, explanatory statements (that comes next). A common mantra in the field of psychology is that correlation does not equal causation. In other words, just because variable A predicts variable B does not mean that A causes B. This is true for two reasons, which we refer to as the
7/30/20, 1)48 PMPrint
Page 22 of 33https://content.ashford.edu/print/Newman.2681.16.1?sections=navp…t&clientToken=517d61d0-db98-8df2-52e3-a6d12160a350&np=navpoint-8
directionality problem and the third variable problem. (See Figure 2.1.)
First, when we measure two variables at the same time, we have no way of knowing the direction of the relationship. Take the relationship between money and happiness: It could be true that money makes people happier, because they can afford nice things and fancy vacations. It could also be true that happy people have the confidence and charm to obtain higher-paying jobs, resulting in more money. In a correlational study, we are unable to distinguish between these possibilities. Or, take the relationship between television viewing and obesity: It could be that people who watch more television get heavier, because TV makes them snack more and exercise less. It could also be that people who are overweight lack the energy to move around and end up watching more television as a consequence. Once again, we cannot identify a cause–effect relationship in a correlational study.
Second, when we measure two variables as they naturally occur, a third variable that actually causes both of them is always a possibility. For example, imagine we find a correlation between the number of churches and the number of liquor stores in a city. Do people build more churches to offset the threat of liquor stores? Do people build more liquor stores to rebel against churches? Most likely, the link involves a third variable, population size, that causes changes in both variables: The more people who are living in a city, the more churches and liquor stores they can support. As another example, imagine a correlation between ice cream sales and homicide rates is discovered. Does ice cream lead people to commit murder? Do murderers like to buy ice cream on the way home from the scene of the crime? Most likely, the link involves a third variable, temperature, that causes changes in both variables: The hotter it gets outside, the more people want ice cream, and the greater likelihood that disagreements will turn violent.
Experimental Research
Finally, recall that research projects can have the goal of attempting to explain a phenomenon. When the research goal involves causal explanations, then research design falls under the category of experimental research, in which the primary goal is to explain thoughts, feelings, and behaviors and to make causal statements. Examples of experimental research questions include:
Does smoking cause cancer? Does drinking alcohol make people more aggressive? Does loneliness cause alcoholism? Does stress cause heart disease? Can meditation make people healthier?
7/30/20, 1)48 PMPrint
Page 23 of 33https://content.ashford.edu/print/Newman.2681.16.1?sections=navp…t&clientToken=517d61d0-db98-8df2-52e3-a6d12160a350&np=navpoint-8
Research: Making an Impact
Helping Behaviors
The 1964 murder of Kitty Genovese in plain sight of her neighbors, none of whom helped, drove numerous researchers to investigate why people may not help others in need. Are individuals selfish and bad, or does a group dynamic lead to inaction? Is there something wrong with our culture, or are situations more powerful than we think?
Among the body of research conducted in the late 1960s and 1970s was one pivotal study that revealed why people may not help others in emergencies. Darley and Latané (1968) conducted an experiment with various individuals in different rooms who communicated with each other via intercom. In reality, the study included just one participant and a number of confederates, one of whom pretended to have a seizure. Among participants who thought they were the only other person listening over the intercom, more than 80% helped, and they did so in less than 1 minute. However, among participants who thought they were one of a group of people listening over the intercom, less than 40% helped, and even then only after more than 2.5 minutes. This phenomenon—that the more people who witness an emergency are present, the less likely any of them is to help—has been dubbed the “bystander effect.” One of the main reasons that this tendency occurs is that responsibility for helping gets “diffused” among all of the people present, so that each one feels less personal responsibility for taking action.
Darley and Latané’s research can be seen in action and has influenced safety measures in today’s society. For example, when someone witnesses an emergency, no longer does it suffice to simply yell to the group, “Call 911!” Because of the bystander effect, we know that most people will believe someone else will do it, and the call will not be made. Instead, it is necessary to designate a specific person to make the call. In fact, part of modern-day CPR training involves making individuals aware of the bystander effect and best practices for getting people to help and be accountable.
Although the bystander effect may be the rule, there are always exceptions. For example, on September 11, 2001, the fourth hijacked airplane was overtaken by a courageous group of passengers. Most people on the plane had heard about the twin tower crashes and recognized that their plane was heading for Washington, D.C. Despite being amongst nearly 100 other people, a few people chose to help the intended targets in D.C. Risking their own safety, this heroic group chose to help to prevent others from experiencing death and suffering. So, although we may see events that remind us of the reality of the bystander effect, we also see moments where people are willing to help, no matter the number of people that surround them.
Think About It:
1. What type of research design best describes Darley & Latane’s (1968) study?
7/30/20, 1)48 PMPrint
Page 24 of 33https://content.ashford.edu/print/Newman.2681.16.1?sections=navp…t&clientToken=517d61d0-db98-8df2-52e3-a6d12160a350&np=navpoint-8
2. What practical applications have resulted from research on people’s reluctance to help in emergencies?
What these five questions have in common is a focus on understanding why something happens. Experiments move beyond, for example, the question of whether alcoholics are more aggressive to whether alcohol actually causes an increase in aggression.
Experimental designs are able to address the shortcomings of correlational designs because the researcher has more control over the environment. Chapter 5 will cover this in great detail, but the basic process of conducting an experiment is relatively simple: A researcher has to control the environment as much as possible so that all participants in the study have the same experience. This helps eliminate other third variables that might influence the results. Researchers will then manipulate, or change, one key variable and then measure outcomes in another key variable. The variable manipulated by the experimenter is called the independent variable (IV). The outcome variable that is measured by the experimenter is called the dependent variable (DV). The combination of controlling the setting and changing one aspect of this setting at a time allows the experimenter to state with some certainty that the changes caused something to happen.
Think of this in a little more concrete way. Imagine that a researcher wanted to test the hypothesis that meditation improves health. In this case, meditation would be the independent variable, and health would be the dependent variable. One way to test this hypothesis would be to take a group of people and have half of them meditate 20 minutes per day for several days while the other half did something else for the same amount of time. The group that meditates would be called the experimental group because it provides the test of the hypothesis. The group that does not meditate would be called the control group because it provides a basis of comparison for the experimental group.
The researcher would want to make sure that these groups spent the 20 minutes in similar conditions so that the only difference would be the presence or absence of meditation. One way to accomplish this would be to have all participants sit quietly for the 20 minutes but give the experimental group specific instructions on how to meditate. Then, to test whether meditation led to increased health and happiness, the researcher would give both groups a set of outcome measures at the end of the study—perhaps a combination of survey measures and a doctor’s examination. If differences were found between the dependent measures for the two groups, the experimenter could be fairly confident that meditation caused them to happen. One way we can operationalize health outcomes in this study would be to measure blood pressure, as higher levels of blood pressure put people at risk for developing cardiovascular disease. So, for example, the researcher might find lower blood pressure in the experimental (meditation) group, which would suggest that meditation causes blood pressure to drop.
Choosing a Research Design
The choice of a research design is guided first and foremost by a researcher’s finding the best fit to the research question and then adjusting it depending on practical and ethical concerns. At this point, a nagging question may come to mind: If experiments are the most powerful type of design, why not use them every
7/30/20, 1)48 PMPrint
Page 25 of 33https://content.ashford.edu/print/Newman.2681.16.1?sections=navp…t&clientToken=517d61d0-db98-8df2-52e3-a6d12160a350&np=navpoint-8
time? Why would anyone give up the chance to make causal statements? One reason is that we are often interested in variables that cannot be manipulated, for ethical or practical reasons, and that therefore have to be studied as they occur naturally. In one example, Matthias Mehl and Jamie Pennebaker (2003) happened to start a weeklong study of college students’ social lives on September 10, 2001. Following the terrorist attacks on the morning of September 11, Mehl and Pennebaker were able to track changes in people’s social connections and use this to understand how groups respond to traumatic events. Of course, it would have been unthinkable to manipulate a terrorist attack for this study experimentally, but since it occurred naturally, the researchers were able to conduct a correlational study of coping.
Another reason to use descriptive and correlational designs is that these are useful in the early stages of a research program. For example, before a psychologist can start to think about the causes of binge drinking among college students, it is important to understand how common is this phenomenon. Likewise, before a researcher designs a time- and cost-intensive experiment on the effects of meditation, it is a good idea to conduct a correlational study to test whether meditation even predicts health. In fact, this latter example comes from a series of real research studies conducted by psychiatrist Sara Lazar and her colleagues at Massachusetts General Hospital. This research team first discovered that experienced practitioners of mindfulness meditation had more development in brain areas associated with control over attention and emotion. But this study was correlational at best; perhaps meditation caused changes in brain structure or perhaps people who were better at integrating emotions were drawn to meditation. In a follow-up study, researchers randomly assigned people either to meditate or to perform stretching exercises for two months. These experimental findings confirmed that mindfulness meditation actually caused structural changes to the brain (Hölzel et al., 2011). This series of studies is a prime example of how a research program can progress from correlational to experimental designs.
Table 2.2 summarizes the main advantages and disadvantages of these three types of design. In addition, the bottom of the table includes two examples of research topics—meditation and health, and temperature and aggression—to showcase the similarities and differences between the designs.
Table 2.2: Summary of research designs
Research Design
Descriptive Correlational Experimental
Goal Describe characteristics of an existing phenomenon
Predict behavior; assess strength of relationship between variables
Explain behavior; assess impact of IV on DV
Advantages Provides a complete picture of what is occurring at a given time
Allows testing of expected relationships; predictions can be made
Allows conclusions to be drawn about causal relationships
Disadvantages
Does not assess relationships; no explanation for phenomenon
Cannot draw inferences about causal relationships
Cannot manipulate many important variables
7/30/20, 1)48 PMPrint
Page 26 of 33https://content.ashford.edu/print/Newman.2681.16.1?sections=navp…t&clientToken=517d61d0-db98-8df2-52e3-a6d12160a350&np=navpoint-8
Example #1: Studying Meditation
What percentage of college students meditate at least once a week?
Are regular meditators happier and healthier?
If we randomly assign people to start meditating, do they become happier and healthier?
Example #2: Temperature and Aggression
How many violent crimes are committed in the summer?
Are crime rates higher in the summer than in the winter?
If we turn up the temperature in the laboratory, do people become more aggressive?
Designs on the Continuum of Control
Before leaving the design overview behind, we will consider how these designs relate to one another. The best way to think about the differences between the designs is in terms of the amount of control a researcher has. That is, experimental designs are the most powerful because the researcher controls everything from the hypothesis to the environment in which the data are collected. Correlational designs are less powerful because the researcher is restricted to measuring variables as they occur naturally. However, with correlational designs, the researcher does maintain control over several aspects of data collection, including the setting and the choice of measures. Descriptive designs are the least powerful because researchers have difficultly controlling outside influences on data collection. For example, when people answer opinion polls over the phone, they might be sitting quietly and pondering the questions or they might be watching television, eating dinner, and dealing with a fussy toddler. As a result, researchers are more limited as to the conclusions they can draw from these data. Figure 2.2 shows an overview of where research designs fall on the continuum of control in order of increasing control: from descriptive, to predictive, to experimental. Chapters 3, 4, and 5 will cover variations on these designs in more detail.
Figure 2.2: The continuum of control framework
7/30/20, 1)48 PMPrint
Page 27 of 33https://content.ashford.edu/print/Newman.2681.16.1?sections=navp…t&clientToken=517d61d0-db98-8df2-52e3-a6d12160a350&np=navpoint-8
7/30/20, 1)48 PMPrint
Page 28 of 33https://content.ashford.edu/print/Newman.2681.16.1?sections=navp…t&clientToken=517d61d0-db98-8df2-52e3-a6d12160a350&np=navpoint-8
5.2 Key Features of Experiments The overview of designs in Chapter 2 described the overall process of experiments in the following way: Researchers control the environment as much as possible so that all participants have the same experience. The researchers then manipulate, or change, one key variable, and then measure the outcomes in another key variable. This section examines this process in more detail. Experiments can be distinguished from all other designs by three key features: manipulating variables, controlling the environment, and assigning people to groups.
Manipulating Variables
The most crucial element of an experiment is researcher’s manipulation, or change, of some key variable. To study the effects of hunger, for example, a researcher could manipulate the amount of food given to the participants, or to study the effects of temperature, the experimenter could raise and lower the temperature of the thermostat in the laboratory. In both cases, recall that the researcher needs a way to operationalize the concepts (hunger and temperature) into measurable variables. For example, the experimenter could define “hungry” as being deprived of food for eight hours, and define a “hot” room as being 90 degrees Fahrenheit. Because these factors are under the direct control of the experimenters, they can feel more confident that changing them contributes to changes in the dependent variables.
Chapter 2 discussed the main shortcoming of correlational research: These designs do not allow researchers to make causal statements. Recall from that chapter (as well as from Chapter 4) that correlational research is designed to predict one variable from another. One of the examples in Chapter 2 concerned the correlation between income levels and happiness, with the goal of trying to predict happiness levels based on knowing people’s income level. If we measure these as they occur in the real world, we cannot say for sure which variable causes the other. However, we could settle this question relatively quickly with the right experiment. Suppose we bring two groups into the laboratory and give one group $100 and a second group nothing. If the first group is happier at the end of the study, it would support the idea that money really does buy happiness. Of course, this experiment is a rather simplistic look at the connection between money and happiness. Even so, because we manipulate levels of money, this study would bring us closer to making causal statements about the effects of money.
To manipulate variables, it is necessary to have at least two versions of the variable. That is, to study the
7/30/20, 1)48 PMPrint
Page 29 of 33https://content.ashford.edu/print/Newman.2681.16.1?sections=navp…t&clientToken=517d61d0-db98-8df2-52e3-a6d12160a350&np=navpoint-8
Monkey Business Images/Monkey Business/Thinkstock
Having a patient run on a treadmill to measure cardiovascular stress is an example of invasive manipulation.
effects of money, we need a comparison group that does not receive money. To study the effects of hunger, we would need both a hungry and a not-hungry group. Having two versions of the variable distinguishes experimental designs from the structured observations discussed in Chapter 3 (3.4), in which all participants receive the same set of conditions in the laboratory. Even the most basic experiment must have two sets of conditions, which are often an experimental group and a control group. However, as this chapter will later explain, experiments can become much more complex. A study might have one experimental group and two control groups, or five degrees of food deprivation, ranging from 0 to 12 hours without food. Decisions about the number and nature of these groups will depend on consideration of both the hypotheses and previous literature.
Researchers have three options for manipulating variables. First, environmental manipulations involve changing some aspect of the setting. Environmental manipulations are perhaps the most common in psychology studies, and they include everything from varying the room temperature to varying the amount of money people receive. The key is to change the way that different groups of people experience their time in the laboratory—it is either hot or cold, and they either receive or do not receive $100.
Second, instructional manipulations involve changing the way a task is described to change participants’ mindsets. For example, a researcher might give the same math test to all participants but to one group, describe it as an “intelligence test” and to another group, a “problem-solving task.” Because an intelligence test is thought to have implications for life success, the experimenter might expect participants in that group to be more nervous about their scores.
Finally, an invasive manipulation involves taking measures to change internal, physiological processes; it is usually conducted in medical settings. For example, studies of new drugs involve administering the drug to volunteers to determine whether it has an effect on some physical or psychological symptom. Alternatively, studies of cardiovascular health often involve having participants run on a treadmill to measure how the heart functions under stress.
The rule that we must manipulate a variable has one qualification. In many experiments, researchers divide participants based on a preexisting difference (e.g., gender) or personality measures (e.g., self-esteem or neuroticism) that capture stable individual differences among people. The idea behind these personality measures is that someone scoring high on a measure of neuroticism (for example) would be expected to be more neurotic across situations than someone scoring lower on the measure. Using this technique allows a researcher to compare how, for example, men and women or people with high and low self-esteem respond to manipulations.
When researchers use preexisting differences in an experimental context, they are referred to as quasi-
7/30/20, 1)48 PMPrint
Page 30 of 33https://content.ashford.edu/print/Newman.2681.16.1?sections=navp…t&clientToken=517d61d0-db98-8df2-52e3-a6d12160a350&np=navpoint-8
independent variables—”quasi,” or “nearly,” because they are being measured, not manipulated, by the experimenter, and thus do not meet the criteria for a regular independent variable. In fact, variables used in this way are things that cannot be manipulated by an experimenter—either for practical or ethical reasons— including gender, race, age, eye color, religion, and so forth. Instead, these are treated as independent variables in that participants are divided into groups along these variables (e.g., male versus female; Catholic versus Protestant versus Muslim).
Because these variables are not manipulated, an experimenter cannot make causal statements about them. For a study to count as an experiment, these quasi-independent variables would have to be combined with a true independent variable. This could be as simple as comparing how men and women respond to a new antidepressant drug—gender would be quasi-independent while drug type would be a true independent variable.
Sometimes the line between true and quasi-experiments can be subtle. Imagine we want to study the effects on people’s persistence at a second task based on winning versus losing a contest. In a quasi-experimental approach, we could have two participants play a game, resulting in a natural winner and loser, and then compare how long each one stuck with the next game. The approach’s limitation is that some preexisting condition might have affected winning and losing the first game. Perhaps the winners had more self- confidence and patience at the start. However, we could improve the design to be a true experiment by having participants play a rigged game against a confederate, thereby causing participants either to win or lose. In this case, we would be manipulating winning and losing, and preexisting differences would be averaged out across the groups (more on this later in the chapter).
Controlling the Environment
The second important element of experimental designs is the researcher’s high degree of control over the environment. In addition to manipulating variables, an experimenter has to ensure that the other aspects of the environment are the same for all participants. For instance, if we were interested in the effects of temperature on people’s mood, we could manipulate temperature levels in the laboratory so that some people experienced warmer temperatures and other people cooler temperatures. However, it is equally important to make sure that other potential influences on mood are the same for both groups. That is, we would want to make sure that the “warm” and “cool” groups were tested in the same room, around the same time of day, and by similar experimenters.
The overall goal, then, is to control extraneous variables, or variables that add noise to the hypothesis test. In essence, the more researchers can control extraneous variables, the more confidence they can have in the results of the hypothesis test. As the section “Validity and Control” will discuss, these extraneous variables can have different degrees of impact on a study. Imagine we conduct the study on temperature and mood, and all of our participants are in a windowless room with a flickering fluorescent light. This environment would likely influence people’s mood—making everyone a little bit grumpy—but it causes fewer problems for our hypothesis test because it affects everyone equally. Table 5.1 shows hypothetical data from two variations of this study, using a 10-point scale to measure mood ratings. In the top row, participants were in a well-lit room; notice that participants in the cooler room reported being in a better mood (i.e., an 8 versus a 5). In the bottom
7/30/20, 1)48 PMPrint
Page 31 of 33https://content.ashford.edu/print/Newman.2681.16.1?sections=navp…t&clientToken=517d61d0-db98-8df2-52e3-a6d12160a350&np=navpoint-8
row, all participants were in the windowless room with flickering lights. These numbers suggest that people were still in a better mood in the cooler room (5) than a warm room (2), but the flickering fluorescent light had a constant dampening effect on everyone’s mood.
Table 5.1: Influence of an extraneous variable
Cool Room Warm Room
Variation 1: Well-Lit 8 5
Variation 2: Flickering Fluorescent 5 2
Assigning People to Conditions
The third key feature of experimental designs is that the researcher can assign people to receive different conditions, or versions, of the independent variable. This is an important piece of the experimental process: Experimenters not only control the options—warm versus cool room, $100 versus no money, etc.—but they also control which participants get each option. Whereas a correlational design might assess the relationship between current mood and choosing the warm room, an experimental design will assign some participants to the warm room and then measure the effects on their mood. In other words, experimenters are able to make causal statements because they cause things to happen to a particular group of people.
The most common, and most preferable, way to assign people to conditions is through a process called random assignment. An experimenter who uses random assignment makes a separate decision for each participant as to which group he or she will be assigned to before the participant arrives. As the term implies, this decision is made randomly—by flipping a coin, using a random number table (for an example, see http://stattrek.com/tables/random.aspx (http://stattrek.com/tables/random.aspx) ), drawing numbers out of an envelope, or even simply alternating back and forth between experimental conditions. The overall goal is to try to balance preexisting differences among people, as Figure 5.2 illustrates. So, for example, some people might generally be more comfortable in warm rooms, while others might be more comfortable in cold rooms. If each person who shows up for the study has an equal chance of being in either group, then the groups in the sample should reflect the same distribution of differences as the population.
Figure 5.2: Random assignment
The 24 participants in our sample consist of a mix of happy and sad people. The goal of random assignment is to have these differences distributed equally across the experimental conditions. Thus, the two groups on the right each consist of six happy and six sad people, and our random assignment was successful.
7/30/20, 1)48 PMPrint
Page 32 of 33https://content.ashford.edu/print/Newman.2681.16.1?sections=navp…t&clientToken=517d61d0-db98-8df2-52e3-a6d12160a350&np=navpoint-8
Forming groups through random assignment also has the significant advantage of helping to avoid bias in the selection and assignment of subjects. For example, it would be a bad idea to assign people to groups based on a first impression of them because participants might be placed in the cold room if they arrived at the laboratory dressed in warm clothing. Experimenters who make decisions about condition assignments ahead of time can be more confident that the independent variable is responsible for changes in the dependent variable.
Worth highlighting here is the difference here between random selection and random assignment (discussed in Chapter 4). Random selection means that the sample of participants is chosen at random from the population, as with the probability sampling methods discussed in Chapter 4. However, most psychology experiments use a convenience sample of individuals who volunteer to complete the study. This means that the sample is often far from fully random. However, a researcher can still make sure that the study involves random assignment to groups, so that each condition contains an equal representation of the sample.
In some cases—most notably, when samples are small—random assignment may not be sufficient to balance an important characteristic that might affect the results of a particular study. Imagine conducting a study that compared two strategies for teaching students complex math skills. In this example, it would be especially important to make sure that both groups contained a mix of individuals with, say, average and above-average intelligence. For this reason, the experimenter would necessarily take extra steps to ensure that intelligence was equally distributed between the groups, which can be accomplished with a variation on random assignment called matched random assignment. This kind of assignment requires the experimenter to obtain scores on an important matching variable—in this case, intelligence—rank participants based on the matching variable, and then randomly assign people to conditions. Figure 5.3 shows how this process would unfold in our math-skills study. First, the researcher gives participants an IQ test to measure preexisting differences in intelligence. Second, the experimenter ranks participants based on these scores, from highest to lowest. Third, the experimenter moves down this list in order and randomly assigns each participant to one of the conditions. This process still contains an element of random assignment, but adding the extra step of rank ordering ensures a more balanced distribution of intelligence test scores across the conditions.
Figure 5.3 Matched random assignment
7/30/20, 1)48 PMPrint
Page 33 of 33https://content.ashford.edu/print/Newman.2681.16.1?sections=navp…t&clientToken=517d61d0-db98-8df2-52e3-a6d12160a350&np=navpoint-8
The 20 participants in our sample represent a mix of very high, average, and very low intelligence test scores (measured 1–100). The goal of matched random assignment is to ensure that this variation is distributed equally across the two conditions. The experimenter would first rank participants by intelligence test scores (top box), and then distribute these participants alternately between the conditions. The end result is that both groups (lower boxes) contain a good mix of high, average, and low scores.