Week 4

Measurement

Video Title: Measurement Originally Published: 2013 Publishing Company: SAGE Publications, Inc.

City: Thousand Oaks, United States ISBN: 9781506358352

DOI: https://dx.doi.org/10.4135/9781506358352

(c) SAGE Publications Inc., 2013 This PDF has been generated from SAGE Research Methods.

[SAGE video in practice] [Measurement]

NARRATOR: Measurement is a part of our everyday lives. Each time we get on a scale or take a

blood pressure reading, we're carefully measuring a variable that's important to us as individuals.

In society, there are also variables that we regularly measure. And such measurement often affects

large numbers of people. For example, the cleanliness of our streets or poverty

NARRATOR [continued]: are important public issues that affect the lives of millions of citizens, affect

the policies we propose and the way tax dollars get spent. Professors Dahlia Remler and Gregg Van

Ryzin are public affairs researchers who note that measurement in the public arena can often be

complex, yet it's essential to good public policy

NARRATOR [continued]: decisions.

DAHLIA REMLER: Government agencies need to measure things, to see what they need to work on

and to see if the programs have worked.

NARRATOR: And that's just what New York City did about clean streets over four decades ago. But

unlike blood pressure or weight--

DAHLIA REMLER: It's multi-dimensional. It's harder to define what actually is street cleanliness.

NARRATOR: So the city began by asking residents what clean streets meant.

SPEAKER 1: No trash on the ground and no clutter in the middle of the street and no smell.

SPEAKER 2: I would say clean street would be free from debris.

SPEAKER 3: A clean street is a street that doesn't have a lot of litter.

SPEAKER 4: You really can't measure it.

NARRATOR: But the city felt it could measure litter. However, precise scientific measurement requires

an operational definition. How would you define litter on the streets?

GREGG VAN RYZIN: Middle of the street, on the curb. Is it on the sidewalk? And how much of that

counts?

NARRATOR: Alternatively, you could measure the number of people hired by the city as street

sweepers or the average amount of time they spend sweeping.

DAHLIA REMLER: So you'd love to measure all of these things to get a really good measure of

the street-- holistic measure of the street cleanliness. That's really expensive and very, very time

consuming.

NARRATOR: And the city already spends over $1 billion per year just cleaning streets. So New York

City made a decision.

GREGG VAN RYZIN: They went through a process of taking photographs.

NARRATOR: The photos were ranked from least clean to most clean.

GREGG VAN RYZIN: Those became a standard, a photographic standard, that was then used to

train raters.

NARRATOR: Today, raters carry handheld phones and compare their judgments to the standards.

But this definition of street cleanliness is very narrow, as it only focuses on litter specifically in the

SAGE 2013 SAGE Publications, Ltd. All Rights Reserved.

SAGE Research Methods Video

Page 2 of 4 Measurement

street as opposed to broadening the definition to include other areas.

DAHLIA REMLER: So we'll have a less valid measure.

NARRATOR: And while the definition may be less valid, less inclusive, the precise but narrow

photographic standard used results in high reliability among raters.

DAHLIA REMLER: Given that you have limited resources, you have to think about this validity-

reliability trade-off.

NARRATOR: And Van Ryzin's research shows that New York City's efforts are working. Over time, as

street cleanliness ratings have risen, so has resident satisfaction.

SPEAKER 5: It's not like the old days you used to walk by and there would be trash all over the place.

And I think, yeah, it's improved a lot since even in the past five years or so, I would say.

NARRATOR: But while street cleanliness has improved in New York City, what's gotten worse, both

here and across the nation, is the poverty rate. This graph shows government statistics on poverty.

GREGG VAN RYZIN: The number of people in poverty over time rising to 46.2 million. But it's very

important to look at how the government defines and measures poverty.

NARRATOR: It was Mollie Orshansky who created the official poverty definition more than half a

century ago.

DAHLIA REMLER: She knew there was a lot of controversy. So she sort of said, well, everyone can

agree that you need to have enough to eat.

NARRATOR: Food cost back then used up about 1/3 of the family's budget.

DAHLIA REMLER: And so they just took that, multiplied it by three, and said, if you have that, you

have enough.

NARRATOR: Amazingly, the definition has changed very little since then.

DAHLIA REMLER: They just took that old thing. And they just updated it and updated it for inflation.

NARRATOR: But since the 1950s, food prices have come down dramatically, while other costs have

risen, causing some critics to be--

DAHLIA REMLER: Stunned and appalled at the official measure. Instead, people want to look at a

genuine measure of necessities, price it, see what it is, agree on that, and use that.

NARRATOR: Today, a supplemental poverty measure addresses several flaws in the official poverty

measure by counting some non-cash sources of income.

DAHLIA REMLER: You can't just look at the income that I've earned and see if I'm poor. You also

have to see if the government gave me food stamps or housing vouchers or other things like that.

NARRATOR: But surprisingly, non-cash health care contributions like Medicare and Medicaid are not

counted.

DAHLIA REMLER: And so I'm concerned, oh, we're not really accurately measuring poverty, because

we're missing out on this important dimension of poverty. That's a validity issue.

NARRATOR: Another issue is whether poverty should be considered absolute or relative.

DAHLIA REMLER: Is poverty a relative concept? Am I poor if I'm so much worse off than other people

SAGE 2013 SAGE Publications, Ltd. All Rights Reserved.

SAGE Research Methods Video

Page 3 of 4 Measurement

no matter what I have? Or do we have an absolute standard for poverty?

NARRATOR: Recall that Orshansky utilized an absolute standard which still applies today.

DAHLIA REMLER: This issue of relative poverty is one that is, I think, quite politically controversial.

And people don't agree about that.

NARRATOR: What we can agree about is that measuring public policy variables can be complex.

And Remler and Van Ryzin have important take-home messages.

DAHLIA REMLER: The first thing to understand is to really think very, very clearly about what you

want to measure.

GREGG VAN RYZIN: Look behind the scenes. See how the measurement is produced. And you

need to be a critical consumer of those sorts of statistics. There are strengths and weaknesses to

almost every measurement that the government or any agency or the media report on. And I think

understanding those strengths and weaknesses is an important skill.

SAGE 2013 SAGE Publications, Ltd. All Rights Reserved.

SAGE Research Methods Video

Page 4 of 4 Measurement