C6

Quantitative Research: Methods in

the Social Sciences

Video Title: Quantitative Research: Methods in the Social Sciences

Originally Published: 2006

Publishing Company: SAGE Publications, Inc

City: Thousand Oaks, USA

ISBN: 9781483397160

DOI: https://dx.doi.org/10.4135/9781483397160

(c) SAGE Publications, Inc., 2006

This PDF has been generated from SAGE Research Methods.

[Quantitative Methods] [Table of Contents-- 1. Questions of Quantitative Research 2. Principles of

Measurement 3. Experiments 4. Surveys 5. Applications 6.Conclusion] [Segment 1 Questions of

Quantitative Research]

NARRATOR: Human behavior is complex. Understanding how, why, and to what ends human beings

do what we do is studied by social scientists through a variety of methods generally referred to as

"quantitative methods." While there are different methods specifically, they each address certain kinds

of questions and adhere to certain principles of measurement.

NARRATOR [continued]: These include questions about cause effect and mitigating effects. What is

the effect of a given cause? What is the cause of a given effect? How do we mitigate a given effect

by manipulating a given cause?

BARBARA HUMMEL ROSSI: Quantitative methods are used when you have specific questions in

mind and good measures to measure the variables in question. For example, you might be looking

at the relation between achievement and intelligence. The question might be, what is the relation

between achievement and intelligence?

BARBARA HUMMEL ROSSI [continued]: Now we have good standardized measures to measure

both intelligence and achievement and we would use correlation analysis to look at the relation

between the two of them.

NARRATOR: The following example illustrates the essence of what quantitative methods seek to

address in whole or part.

X: I was trying to call you Saturday and you didn't pick up. Where were you?

Y: Oh, yeah. I was out, just out with some friend of mine.

X: Where'd you go? What'd you do?

Y: Just to a bar. I was just hanging out with a girl named Sally. Yeah.

X: Who is she?

Y: It was just kind of like a date.

X: OK. So let me just try to get this straight. You went out with her Saturday night on a date without

even telling me, without even letting me know. And you apparently like her more than you do and now

you're breaking up with me. Well, just try for the sake of knowing things, I just want to know what you

did with her. What went on that you're keeping from me?

Y: It doesn't matter.

X: No, to me, it matters.

Y: It doesn't.

X: I want to know what you did with her behind my back. That's what I want to know.

Y: It's not about that.

NARRATOR: Those using quantitative methods to understand what happened between these two

people would want to know, what is X feeling? Did what Y said to X make her upset? If Y would have

said something more positive, would X be expressing a different emotion? [Segment 2 Principles of

SAGE

2006 SAGE Publications, Ltd. All Rights Reserved.

SAGE Research Methods Video

Page 2 of 8 Quantitative Research: Methods in the Social Sciences

Measurement]

NARRATOR [continued]: When measuring these various causes and effects, social scientists are

careful that they measure what and how things occur in the real world, not the world as it exists in their

office, laboratory, or their own brain. This includes adhering to standards of internal validity, external

validity, and reliability.

NARRATOR [continued]: Internal validity is when an experiment isolates a causal connection

between two variables, eliminating all other explanations. External validity is when results of a

study can be generalized to a broader population. Reliability is when a phenomenon is measured

consistently

NARRATOR [continued]: in repeated studies.

BARBARA HUMMEL-ROSSI: Internal and external validity are really both critical for doing

experiments, particularly the experimental control situation. Internal validity refers to, does the

treatment make a difference? And you'd be concerned about such things interfering with the

treatment effect, such things as history.

BARBARA HUMMEL-ROSSI [continued]: As a person gets older, the construct under question may

change. You would be concerned about the effects, for example, of a pretest sensitizing the individual

to the intervention and the effects perhaps of differential mortality, that is, people leaving

BARBARA HUMMEL-ROSSI [continued]: the experiment differently in the control group and the

experimental group. With respect to external validity, this has to do with whether or not you can

generalize to other situations, for example, to another setting, to other people administering an

intervention. And they're both very critical to experimental design.

BARBARA HUMMEL-ROSSI [continued]: [Segment 3 Experiments]

NARRATOR: One of the most often-used forms of quantitative methods is the experiment.

CHARLES MCILWAIN: The primary reason that experiments are used in social science research

is because it's the best method for isolating causal relationships between human behavior. So for

instance, say I wanted to understand whether or not people's attitudes about crime are changed by

the amount or the kind of television news

CHARLES MCILWAIN [continued]: that they watch. An experiment allows the researcher to

manipulate the message, to measure the effect of people's attitudes and opinions, and then be able

to tell whether or not the message was the actual cause of the change in their attitude or their opinion.

The one downside about using experiments

CHARLES MCILWAIN [continued]: is that it is low in what social scientists refer to as external validity.

And that simply means that an experimental environment, the researcher controls everything that's

going on. And we know that in the real world, we don't always know what's going to happen. And so

though we can test for the causal relationship,

CHARLES MCILWAIN [continued]: we can't always generalize to say that this is the way things are

likely to happen in any given scenario.

SAGE

2006 SAGE Publications, Ltd. All Rights Reserved.

SAGE Research Methods Video

Page 3 of 8 Quantitative Research: Methods in the Social Sciences

NARRATOR: The following example illustrates how a typical social science experiment might be run.

This one seeks to ascertain the effects of racial messages in political campaign advertisements. First,

the experimenter describes to subjects in the experiment what they will be doing and asks for their

voluntary consent

NARRATOR [continued]: to continue participation.

CHARLES MCILWAIN: Please sign the form and I will collect them.

NARRATOR: Second, participants are asked to watch a series of political ads in which no racial

message is present.

DAVID JACKSON: What choice do you have in this election? You can choose a candidate who

believes parents should choose whether children will get the best education, instead of being forced

into failing schools. Or you can choose a candidate whose education plan means simply throwing

more money at schools and teachers who aren't getting the job done. You can choose a candidate

who believes that the way

DAVID JACKSON [continued]: to strengthen our schools is to impose the tough standards of No Child

Left Behind. Or you can choose one who rewards failing teachers and schools who don't meet high

standards of excellence. You have a crucial choice in this election. I'm David Jackson and I want to

be your choice because I'm the right choice.

NARRATOR: Third, participants are asked to fill out a brief questionnaire that asks, among other

things, how strongly they felt about each candidate and who they would most likely vote for. This

establishes a baseline to measure the effect of the messages to come. Next, the researcher repeat

steps one and two

NARRATOR [continued]: with a different group of participants. These participants then also view a

series of ads. This time, the ads have an explicit racial appeal.

JIM HERBERT: Some people have said that the difference between my opponent and me is the color

of our skin. That's not the only difference. David Jackson's education plan is to take money away

from folks like us to fund inner city schools that look like him. Jackson says his quota-based so-called

affirmative action in education plan is necessary to make the children in our two

JIM HERBERT [continued]: communities more equal. Jackson is a good man and we both believe in

equality. But does equality mean that it's fair to take money from one group and give it to another just

because of the color of their skin? I'm Jim Herbert and I'm running for Congress because I believe in

an education policy that isn't just black and white.

NARRATOR: Next, subjects are again asked to fill out a questionnaire that asks the same questions

about how they felt about each candidate and which of them they would more likely vote for. After this,

the experimenter analyzes data to see if there was a measurable difference in participants' attitudes

between those who saw ads with no racial message and those who saw ads

NARRATOR [continued]: with explicit racial messages. In this brief example, the researcher

conducting the experiment will analyze the data, hoping to determine whether there is a causal link

SAGE

2006 SAGE Publications, Ltd. All Rights Reserved.

SAGE Research Methods Video

Page 4 of 8 Quantitative Research: Methods in the Social Sciences

between a person's exposure to racial messages and their perception of and likelihood to vote for a

particular political candidate.

CHARLES MCILWAIN: Conducting this experiment allowed us to find out a variety of interesting

conclusions regarding the way that racial messages affect voters. Most importantly, we found that

implicit racial messages seem to work well, in that when voters were exposed to a racial message or

an implicit racial message

CHARLES MCILWAIN [continued]: by a white candidate, they tended to view that candidate more

favorably than the black opponent. However, we also found that explicit racial messages seem to

backfire on the sponsor of the message so that the white candidate who used an explicit racial

message, the voters tended to view that person more negatively

CHARLES MCILWAIN [continued]: and the black opponent more positively. So we can see these two

outcomes as far as how these messages affect the attitudes and beliefs of the voters about these

candidates. But remember, when we're talking about experiments in particular, we're interested in

causation. What is the precise cause for the attitude

CHARLES MCILWAIN [continued]: change in these voters? And in this way, we found that more than

the message itself, there was a greater predictor or causal variable for this attitude change and here,

that was political ideology. So a voter's particular way of seeing political issues

CHARLES MCILWAIN [continued]: had a greater predictive effect or greater causal effect on their

attitude change. [Segment 4 Surveys]

NARRATOR: Surveys are another form of quantitative method used by social scientists. We are all

familiar with and probably have responded to surveys that seek to measure everything from public

opinion on political issues to our use of commercial products to worker satisfaction with their jobs. All

surveys are the same, in that

NARRATOR [continued]: they seek information that allows researchers to probe the depth and/or

breadth of human attitudes and behaviors. However, they can be administered in different ways,

as questionnaires or interviews. Surveys seek to gain quantitative data about a large number of

individuals' opinions

NARRATOR [continued]: or experiences. In questionnaires, individuals respond to written items that

ask them to self-report their attitudes and behaviors. In interviews surveys, a living person administers

a survey face-to-face to individuals, allowing a researcher to clarify responses.

INTERVIEWER: Of using surveys--

JACQUELINE MATTIS: In the social sciences, surveys are used as a way of providing broad

descriptions of phenomena. So for example, we have interest in describing patterns of illness, the

rates of incidence of certain kinds of events, and surveys provide a wonderful opportunity to get raw

data on a number of different people and a number of different experiences and from people

JACQUELINE MATTIS [continued]: from different regions, locations, and social environments, et

cetera. But there are naturally positives as well as negatives with using survey research as a

SAGE

2006 SAGE Publications, Ltd. All Rights Reserved.

SAGE Research Methods Video

Page 5 of 8 Quantitative Research: Methods in the Social Sciences

methodology for getting access to information. On the positive end, surveys provide, again, wonderful

opportunities to get data on a number of different people. Because they're so easy to administer,

JACQUELINE MATTIS [continued]: you have opportunities to reach people that you probably

wouldn't be able to reach if you were using other technologies. On the negative end, there are a

number of different concerns with using surveys as a way of going about getting access to data.

First of all, researchers often devise surveys on their own without a great deal of discussion with

participants about their experiences,

JACQUELINE MATTIS [continued]: about the way that they make sense of events, et cetera. And in

those situations, the researcher's ideas and biases very often make themselves known and manifest

in the way that we ask questions, the specific questions that we ask, et cetera. Also on the negative

end, we have with surveys an interesting tendency to assume

JACQUELINE MATTIS [continued]: that the kinds of questions that we can ask are broad enough and

detailed enough to really access a particular phenomenon. So we assume, for example, that if we're

asking questions about depression, that depression means the same thing to all people, which may

not necessarily be the case. We assume that in 10 or 15 or 20 questions,

JACQUELINE MATTIS [continued]: we can get access to the full phenomenon that we describe

as "depression." And it's really difficult sometimes to know whether or not we're asking enough

questions, whether we're asking detailed enough questions, et cetera. So there are always

drawbacks to using surveys as the way that we access information.

NARRATOR: We've seen people respond on their own to a questionnaire. In the following example,

however, the researcher is administering the survey in a structured one-to-one interview.

CHARLES MCILWAIN: First of all, if you are able to vote in the election between David Jackson and

Vincent Fox, who would you be most likely to vote for based on what you know of the two candidates?

Would you say you would vote for David Jackson or Vincent Fox?

WOMAN: David Jackson was the first?

NARRATOR: Notice how more nuanced information might be gained from this method of surveying.

In surveys, issues of validity and reliability have to do with three primary areas, sampling the process

of selecting survey respondents in order to generalize findings, question selection-- questions

NARRATOR [continued]: are designed to elicit the desired information and are relatively free of bias--

and administration. Questions are asked consistently and in the same manner as an interview survey.

JACQUELINE MATTIS: Sampling is the term that we use to describe the people we choose to include

in a particular study. It also describes the places-- the events that we choose to examine. So one of

the more important things to consider when it comes to making decisions about to whom we want to

address the surveys that we're

JACQUELINE MATTIS [continued]: interested in in distributing is the question of, what's the

phenomenon that's of interest here? And are there certain people who are more likely to experience

SAGE

2006 SAGE Publications, Ltd. All Rights Reserved.

SAGE Research Methods Video

Page 6 of 8 Quantitative Research: Methods in the Social Sciences

a phenomenon than others? So for example, if women are more likely to experience a phenomenon

than men are, then we want to focus our attention on women.

JACQUELINE MATTIS [continued]: If we want to examine whether or not something is representative

for African Americans versus Asian Americans, we want to make sure that we sample enough people

from those different backgrounds to make sure that what we're getting is a broad enough overview

of the phenomenon of interest than we would normally get if we only focused on certain groups of

people.

JACQUELINE MATTIS [continued]: The wording that we use in developing any particular question

is extremely important. One thing that we know from doing social science research is that people

use very different language to describe their experiences. So what one person might describe as

"depression" another person may describe using completely different language.

JACQUELINE MATTIS [continued]: And if we assume that one particular word or kind of wording is

representative of an experience for everyone, we often are making mistakes. So the wording of a

question is extremely important. We also have to consider in constructing items whether or not we

have enough questions to really capture a phenomenon that we're interested in.

JACQUELINE MATTIS [continued]: So sometimes, one question is perfect. In many situations,

especially when it comes to the social sciences, we need multiple questions to get at various aspects

of an experience. So again, if we're looking at depression, depression includes emotions. It includes

behaviors. It includes thoughts. And so we want to make sure that we have enough questions that

JACQUELINE MATTIS [continued]: get at emotions or thoughts or behaviors to really capture the

phenomenon as well as we can. We also have to consider issues like ethnicity and language. And

so the wording of a particular question, certain ethnic groups may use certain ways of describing or

discussing a phenomenon and we have to be sensitive to that in the way

JACQUELINE MATTIS [continued]: that we word the items on any particular survey. The final issue

that we have to keep in mind is the order in which items may appear on a survey. If you ask an item

that will bias people to think a certain way or to experience a certain thing, you want to make sure

that you ask those kinds of questions late in a survey rather than early

JACQUELINE MATTIS [continued]: so that you don't bias people too early on in the process.

[Segment 5 Applications]

CHARLES MCILWAIN: One of the areas of application for quantitative methods is in the area of

marketing, where the makers or producers of products try to understand what it is their audience or

consumers want. And so they seek to measure what it is those people desire,

CHARLES MCILWAIN [continued]: as well as being able to understand how to best persuasively

target that market in order to consume those products. And so quantitative methodology is used

in this area to be able to see whether or not a persuasive message is working. Is it successful in

persuading consumers to buy a particular product or set of products?

CHARLES MCILWAIN [continued]: Another application of quantitative methodology is in assessing

SAGE

2006 SAGE Publications, Ltd. All Rights Reserved.

SAGE Research Methods Video

Page 7 of 8 Quantitative Research: Methods in the Social Sciences

social programs. For instance, if a government or a private agency is trying to set up a program aimed

at curing a social problem, let's say for example, drug abuse, where participants might come into a

treatment program, I want to be able to measure

CHARLES MCILWAIN [continued]: at the end of that program the success or failure of the treatment.

And so quantitative methods are used in this particular instance to be able to look at the end and say,

was this particular treatment effective in declining the drug use or dependency of drug users in these

situations?

CHARLES MCILWAIN [continued]: Another area where quantitative methods are applicable is in the

area of government and politics because in this country in particular and others, the government is

supposed to be responsive to the people that it represents. Government officials, politicians, often

seek to understand what citizens' attitudes are, what

CHARLES MCILWAIN [continued]: their beliefs about particular public policies are so that they can be

more responsive to those needs. And so often, quantitative surveys and so forth are used to measure

public opinion, how different groups of people view a particular social issue or political policy, what it

is that they want or don't want

CHARLES MCILWAIN [continued]: or what they expect or don't expect from their government. And

measuring these allows then government officials and politicians to again be more responsive or at

least to know what it is their citizens want. Another example in which quantitative methods are applied

is in the workplace. Business owners, owners of companies,

CHARLES MCILWAIN [continued]: often know and realize that the success of their company, the

success of their product, is in having a workforce, having employees that are satisfied with their work,

satisfied with the physical conditions of their workplace, satisfied and motivated about the products

that they're selling or the services that they are giving.

CHARLES MCILWAIN [continued]: And so often, employers use quantitative methods, surveys, focus

groups in order to measure what we call "job satisfaction," to be able to tell whether or not their

employees are indeed getting what it is that they need from their work and in turn, the degree that

they're allowing that to be channeled

CHARLES MCILWAIN [continued]: into the selling of their particular product. [Segment 6 Conclusion]

NARRATOR: Whether using experiments, surveys, or a variety of other possible methods,

quantitative researchers or social scientists are able to find causal connections between human

behavior or make inferences about how human beings act, think, and feel about their everyday

actions and interactions with others.

SAGE

2006 SAGE Publications, Ltd. All Rights Reserved.

SAGE Research Methods Video

Page 8 of 8 Quantitative Research: Methods in the Social Sciences