Assignment: Research Designs

\ A / henever there is a high profile crime or victimization, journalists typi- V V cally scramble to the scene to put a human face on the story or turn to

bystanders for their reactions. For example, after four members of a Brooklyn family, including a toddler and a teenage boy, were found shot to death on March 14,2018, Tlte I'{ew York Times reporters sought insight from the neighbors about the alleged killer, Terrance Briggs, who took his own life after killing the other three. Responses to these queries are often inconsistent. For example, one neigh- bor stated that the family argued and that Briggs "liked to play the first-person shooter video game Call of Duty on his P1ayStation," while another said, "This is shocking to me. They were friendly people, nothing bad about them" (Mueller, Piccoli, and Southall 2018). While these person-on-the-scene or on-the-street intenriews are interestirg and provide personal.naruatives to stories, they do not tell us much more than anecdotal information. In other words, we dont know how generalizable or reliable these impromptu interviews are.

In this chapter, you will learn about sampling methods, the procedures that primarily determine the generahzability of research findings. Wb first review the rationale for using sampling in research and consider two circumstances when sampling is not necessary. The chapter then turns to specific sampling methods and when each is most appropriate. This section is followed by

^ sec-

tion on sampling distributions, which introduces you to the logic of statistical inference-that is, how to determine how likely it is that sample statistics rep- resent the population from which the sample was drawn. By the chapter's end, you should understand which questions you need to ask to evaluate the gener- ahzabtlity of a study as well as what choices you need to make when designing a sampling strategF. You should also rcalize that it is as important to select the right people or objects to study as it is to ask participants the right questions.

SAMPLE PLANNING

You have encountered the problem of generaLrzability in each of the studies you have read about in this book. For example, MacDonald et al. (2005) general- ized their sample-based explanation of adolescent aggression and violence to the population of high school students in the United States; Esbensen and his colleagues (2013) generaLized their evaluation findings of violence prevention programs from several middle schools to all middle schools; and Sherman and Berk (1984) and others (Berk, Campbell, Klap, and Western L992; Dunford,

Learni*g Objectives 1.. Identify the circumstances

that make sampling unnecessary and the reason they are rare.

2. Identify the relation among the desired sample, the obtained sample, the sampling frame, and the sample quality.

3. Define and distinguish between probability and nonprobability s ampling and both techniques' relationship to sample generalizability.

4. Def,ne the major types of pfobability sampting and indicate when each is preferred.

nonprob ability samplin g methods may be preferred.

D.esi.ribe,.,the.e6.ne.ept.,of',,,,',

sampling error and explain how..its size iS.:..a.fft.Cted b, : the heterogeneity of the population and the fraction of popnlation inCtuded in : the sample.

fi:;',.N:',7,

Population:

Thr entire set *f e ierne nts ie,u,, indiviriuais, ritirs, states, rtunirios, pri$*n$,

**hrois) in v,,hicl: '';r* are rnte rrsterJ,

Sample:

& substt *f *lerrt*rrts f r*rr th* larrSxr pop u I ati L'in,

Elements:

Th* indiiiirl*al rrternh*rs rf tho p*pLrlalirsn ''i,rh*s* r:haract*ri *ii cs ar * i.rs'*s m*asur"*ri,

Sampling frame:

A list cf ih* *l*r**r$* rsl a potr]ulation f roni ri,rhi*h a

*a.mpk a*tisally ir sclcctcrl.

Enumeration units:

lJnits that r;*rrtain r:n* r:r

rnore *l*mxnts and that ar*

Ni*t*d in a sampling tran*,

SamplinU units:

The uirifs a*tually re le rted in *,trh sta$* r:f tarni:iinu,

Huizin ga, and Elliott 1990; Garner, Fagan, and Maxwell 199 5 ; Hirschel, Hutchison, and Dean 1992; Pate and Hamilton 1992; Sherman et al. 1992) tried to determine the gen eruliz- ability of findings from the original study of domestic violence in Minneapolis. fu sampling is very common in social research, whether we are designing a sampling strategy or evaluating the generalizabiliry of someone else's findings, we have to under- stand how and why researchers decide to sample.

The Purpose of Sampling The purpose of sampling is to generate a set of individuals or other entities that give us a valid picture of all such individuals or other entities. That is, a sample is a subset of the larger set of individuals or other entities in which vr'e are interested. If we have done a good job of sampling, we will be able to generalize what we have learned from the subset to the larger set from which it was selected.

As researchers, we call the set of individuals or other entities to which we want to be able to generalize our findings the population For example, on April 25,2015, over 1,000 people marched to the Baltimore City Hall to protest the death of Freddie Gray, who died of a spinal cord injurywhile in the custody of the Baltimore Police Departrnent. Suppose we wanted to understand their motivations. It would be virnrally impossible to interview all protesters. The entire group would be the population. Instead, we would likely interview a subset of the protesters, which is called a sample. The individual members of this sample are called elemerrtsor elerztmtary units.

Define Sample Components and the Population In many studies, we sample direcdy from the elements in the population of interest. We may survey a sample of the entire population of students in a school based on a list obtained from the registrar's office. This list, from which the elements of the population are selected, is termed the sampling fiame.The students who are selected and interviewed from that list are the elements.

In some studies, the entities that can be reached easily are not the same as the elemens from which we rmnt information, but they include those elements. For example, we may have a Iist of residential addresses but not a Iist of the entire population of a town, even though the aduls in the town are the elements that we actually want to sample. In this situation, we could draw a sample of households, so we canidentitrthe adultindividuals in these households.The households are termed enumeration unie, and the aduls in the households are the elemens (-evy and Lemeshow 1999).

In other instances, the individuals or other entities from which we collect information are not actually the elements in our study. For example, suppose we are interested in finding out the availability of substance abuse treatrnent programs in state prisons. To do this, we might first select a sample of prisons. From within those selected prisons, we might interview a sam- ple of inmates in each prison to obtain information about substance abuse treatrnent program availability. In this case, both the prisons and the inmates are termed sampling units, because we sample from both Q-evy artd Lemeshow 1999). The prisons are selected in the first stage of the sample, so they are the prim.ary sampling units. (n this case, they are also the elements in the study.) The inmates are second.ary sampling units. (Brt they are not elemens, because they are used to provide information about the entire prison.) (See Exhibit 5.1.)

It is important to know exactly what population a sample can represent when you select or evaluate sample components. In a survey of "adultAmericars," tlle general population may reasonably be construed as all residents of the United Sates who are at least 18 years old. But always be alert to ways in which the population may have been narrowed by the sample selec- tion procedures. For example, if the survey was conducted in English only, it would represent

118 SECTION II . FUNDAMENTALS OF RESEARCH

Sample of Prisons

Prisons are the elements and the primary sampling unit.

Sample of lnmates in the Prisons

lnmates are the secondary sampling units; they provide information

about the prisons.

only English-speaking residents of the United States. The population for a study is the aggre- gation of elements that we actually focus on and sample from, not some larger aggregation that we really wish we could have studied.

Some populations, such as the homeless, are not identified by a simple criterion such as a geographic boundary or an organizational membership. Let's say we were interested in victimizations experienced by the homeless population. In this case, a clear definition of the homeless population is difficult but quite necessary. In research, anyone should be able to determine what population was actually studied. FIowever, early studies of home- less persons'odid not propose definitions, did not use screening questions to be sure that the people they interviewed were indeed homeless, and did not make major efforts to cover the universe of homeless people" (Bvt 1996, l5). For example, some studies relied on homeless persons in only one shelter. The result was "a collection of studies that could not be compared" (15). Several studies ofhomeless persons in urban areas addressed the problem by employing a more explicit definition of the population: People are homeless if they have no home or permanent place to stay of their own (renting or owning) and no regular arrangement to stay at someone else's place. Even this more explicit definition still leaves some questions unanswered: What is a regular arrangement? How permanent does a permanent place have to be? The more complete and explicit the definition of the population from which a sample is selected, the more precise our generalizations from a sample to that population can be.

CHAPTERs o SAMPLING 119

Exhibit 5.1 Sample Gomponents in a TWo-Stage Study

Sampling error:

Ariy dill*r*n** b*tv,rc*it tft* $tarat:'t*ri sti rs *t a *amysl* arnil tlt* r;hara*l*risti cs *f ti:c p*pulati** f r*ru lvhich it ruts i$a\Nfi Tfrt,inr{}*r

lh* *r,mfrling *rrrlr,tlt* 1***

af th* iloilrllaiirtn.

Target population:

*, *r;t rst *l*n*nt* larg*r t*a* r:r r:il{*r**i trrsm tlz*

{}{}l}lil at i r: n **.ntSsl *d arsd trt vthi*h ths r*sca r*h*r w*xld lik* tr firnrr*iize rtudy findi*us,

Eval uate General izabi I ity After we clearly define the population we will sample, we need to determine the scope of the generalizations we will seek to make from our sample. Let us say we were interested in the extent to which high school youth are fearful of being attacked or harmed at school or while going to and from their schools. It would be easy to go down to the local high school and hand out a survey asking students to report their levels of fear in these situations. But what if our local high school was located in a remote and rural area of Alaska? Would this sample reflect levels of fear perceived by suburban youth in California or urban youth in New York City? Obviously not. Often, regardless of the sample utilized, researchers will go on to talk about how "this percentage of high school studens is fearfril," or "freshman students are more fearfirl than seniors," as if their study results represent all high school students. Many researchers (and most everyone else, for that matter) are eager to draw conclusions about all individuals they are interested in, not only their samples. Generalizations make their work (and opinions) sound more important. If every high school student were like every other one, generalizations based on observations ofone high school student would be valid. But ofcourse, that is not the case.

As we noted in Chapter 2, geteralizability has two aspects. Can the findings from a sam- ple of the population be generalized to the population from which the sample was selected? Sample generalizability refers to the ability to generalize from a sample, or subset, of a larger population to that population itself. This is the most common meanin g of generalizability. Carr the findings from a study of one population be generalized to another, somewhat different population? This is cross-popalation gmeralizability and refers to the ability to generalize from findings about one group, population, or setting to other groups, populations, or settings (see Exhibit 5.2). In this book, we use the term external aalidity to refer only to cross-population generalizability, not to sample generalizability.

Generalizability is a key concern in research design. We rarely have the resources to study the entire population that is ofinterest to us, so we have to select cases to study that will allow our findings to be generalized to the population of interest. We can never be sure that our propositions will hold under all conditions, so we should be cautious in generalizing to populations that we did not actually sample.

This chapter primarily focuses on the problem of sample generalizability: Can findin$s from a sample be generalized to the population from which the sample was drawn? This is the most basic question to ask about a sample, and social research methods provide many tools to address it.

Sample generalizabfity depends on sample quality, which is determined by the amount of sampling error. Sampling error can generally be defined as the difference betureen the duracteristics of a sample and the characteristics of the population from which it was selected. The larger the sampling error, the less representative the sample and thus the less generalizable the findings. To assess sample quality when you are planning or waluating a study, askyourself these questions:

o From what population were the cases selected? o What method was used to select cases from this population? . Do tlle cases that were studied represent, in the aggregate, the population from which

theywere selected?

In reality, researchers often project their theories onto groups or populations much larger than, or simply different from, those they have actually stufied. The target population is a set of elements larger than or different from the population drat was sampled and to which the researcher would like to generulize any study findings. When we generalize findings to target populations, we must be somewhat speculative. We must carefiilly consider the claim that the findings can be applied to other groups, geographic areas, cultures, or times.

Because the validity of cross-population generalizations cannot be tested empirically, except by conducting more research in other settings, we do not focus much attention on

120 SECTION II . FUNDAMENTALS OF RESEARCH

...we can generalize the sample results to the population from which the sample was selected...

lf we pull a representative sample from a population...

...but we should be cautious in generalizing to another setting or population.

this problem here. We will return to the problem of cross-population generalizability in Chapter 7, which addresses experimental research.

Assess Population Diversity: Research in a Diverse Society

Sampling is unnecessary if all the units in the population are identical. Physicists do not need to select a representative sample ofatomic particles to learn about basic physical processes. They can study a single atomic particle, because it is identical to every other particle of its type. Similarly, biologists do not need to sample a particular type of plant to determine whether a given chemical has toxic effects on it. The idea is "if you've seen one, you've seen 'em all."

What about people? Cerainly all people are not identical, nor are animals in many respects. Nonetheless, if we are studying phpical or psychological processes that are the same among all people, sampling is not needed to achieve generalizable fiodi"S.. Psychologiss and social psychologists often conduct experiments on college students to learn about processes that they think are identical for all individuals. They believe that most people will have the same reactions as the college students if they experience the same experimental conditions. Field researchers who observe group processes in a small community sometimes make the same assumption.

121

,,,,ffi. )ffiffiffi

CHAPTER 5 . SAMPLING

Exhibit 5.2 Sample and Gross-Population Generalizability

Bepresentative sample:

fr sr:mpl*thnt l**ks itk*;lt* ^ ir\t ti.rii,.., i u'r.,ur rr,lri r.l. i+,o,n r.VVUJ ifl il U l t, l1 1Jll\ i$ 111',:i1 li \iV <Ll t:ttlr-rlo,'{ in *il rc'*r,*r.1,: i'i^':ti \.i \J | \.J U | \) L1 i 1 I t^ t | \ r.1'..) f) 7-r W'. t._1 t.2 I 1.t I

?)t* {}*t*tttii'JlV f *l*u?)*i l:{j +1,,, ^t,,;.,"f L,^ ,]i,,;,-'.L,.oi,,", ,.{iltH .-J 1:tlil t llA tl!\ia l1t!l1Jr\tJ t11 ,t11, ij!,Lt\-r\i 1 l1\-J LIt1.)1.1 llljJ1l\.til t!1

Ct?.{'},tllp,i i gli r:s ?{filtr}g T?i*

".1 ".r-," ". r"it,,,I n r'n^. r,t,t r,r^,iei., t *

'-ll+1llt+ltl i t1! /i I l',1 ll tst\k;rli /!l.ttt'.1,, i l)11 i\)lJl\) g. L. 1 \ll) I \J\t\illllAlr'i \l

$;iflril1* it i,lt* i'attt*: z'i ilt* ais,-ri*Ltltrsrr {it l**tit t I t,:t r' 2 r.t *.ri +ti f' r: :] r r, t\ fi fi'\!tlt)11,1\.JL\.; { l!)LI-.1 .l *ttlr.Jtl'*

ih* t*ii.tl #fi'fililtlirl't,*i i:* '11 it { t ii r y, li e nt a liti *,ci fi.{ riiil e,

*{}rif} *har a*irt r i *!i t: s ri r' f\ (\i M 1' o'(\ /7 r' r'\ i7 + * 1' * tl * r'/'11 t- llirf t i r, :J f\tiiifl I t\lv.t \-t v r!,r'vF.r. rrtJvt

* m} e r r *p t' *,q * *! * tj,'an d

Sr\{li'ill lt1 tJ *t { t)l ti{\,}rii *,i 3

There is a potential problem with this assumption, however. There is no way to know ifthe processes being studied are identical for all people. In fact, experiments can give dif- ferent results, depending on the qpe of people studied or the conditions for the experiment. Milgram's (1965) classic experiments on obedience to authority, among the most replicated experiments in the history of social psychological research, illustrate this point very well. The Milgram experiments tested the willingness of male volunteers in New llaven, Connecticut, to comply with instructions from an authority figure to give electric shocls to someone else, even when these shocks seemed to harm the person receiving them. In most cases, the volun- teers complied. Milgram concluded that people are very obedient to authority.

Were these results generalizable to all men, to men in the United States, or to men in New Haven? We have confidence in these findings because similar results were obtained in many replications of the Milgram experiments when the experimental conditions and sub- jects were similar to those studied by Milgram. Other studies, however, showed that some groups were less likely to react so obediently. Given certain conditions, such as another sub- ject in the room who refused to administer the shocks, subjects were likely to resist authority.

So, what do the experimental results tell us about how people will react to an authori- tarian movement in the real world, when conditions are not so carefirlly controlled? In the real social world, people may be less likely to react obediendy. Other individuals may argue against obedience to a particular leader's commands or people may see on TV the conse- quences of their actions. But alternatively, people may be even more obedient to authority than the experimental subjects, as they get swept up in mobs or are captivated by ideological fervor. Milgram's research gives us insight into human behavior, but there is no guarantee that what he found with particular groups in particular conditions can be generalizedto the larger population (or to any particular population) in different settings.

Accurately generalizing the results of experiments and of participant observation is risky, because such research often studies a small number ofpeople who do not represent a particu- lar population. Researchers may put aside concerns about generalizabilitywhen they observe the social dynamics of specific clubs or college dorms or when they conduct a conrolled experiment that tests the effect of, say a violent movie on feelings for others. Nonetheless, we should still be cautious about generalizing the results ofsuch studies.

But what ifyour goal is not to learn about individuals but about the culture or subculture in a society or group? The logic of sampling does not apply if the goal is to learn about culture that is shared across individuals:

When people all provide the same information, it is redundant to ask a question over and over. Only enough people need to be surveyed to eliminate the possibility of errors and to allow for those who might diverge from the norm. (Ileise 2010, 15)

Ifyou are trying to describe a group or society's culture, you may choose individuals for the survey based on their ftnowledge of the culture, not as representatives of a population of individuals (IIeise 2010). In this situation, what is imporant about the individuals surveyed is what they have in common, not their diversity.

Keep these exceptions in mind, but the main point is that social scientists rarely can skirt the problem of demonstrating the generalizability of their findings. If a small sample has been studied in an experiment or a field research project, the study should be replicated in different settings or, preferably with a representative sample of the population to which generaliza- tions are sought (see Exhibit 5.3). The social world and the people in it are too diverse to be considered idmtical units in most respects. Social psychological experiments and small field studies have produced good social science, but they need to be replicated in other settings, with other subjects, to claim any generalizability. Even when we believe that we have uncov- ered basic social processes in a laboratory experiment or field observation, we should be very concerned with seeking confirmation in other samples and in other research.

FUNDAMENTALS OF RESEARCH122 sECTtoN n

Population: 33% (5 out of 15)

satisf ied

Representative sample: 33% (2 out of 6) satisfied

Unrepresentative sample: 66% (4 out of 6) satisfied

Consider a Gensus

In some circumstances, researchers can bypass the issue of generalizability by conducting a census studying the entire population of interest rather than drawing a sample. The federal government tries to do this every 10 years with the U.S. Census. A census can also include studies of all the employees (or students) in small organizations (or universities), studies com- paring all 50 states, and studies of the entire population of a particular type of organization in a particular area. IIowever, in all these instances (except for the U.S. Census), the population studied is relatively small.

The reason that social scientists don't often attempt to collect data from all the members ofsome large populationis simplythat doingso would be too expensive and time-consuming- and they can do almost as well with a sample. Some social scientists conduct research with data from the U.S. Census, but the goveflrment collects the data and our tax dollars pay for the effort to get one person in about 134 million households to answer 10 questions. To conduct the 2010 census, the U.S. Census Bureau spent more than $5.5 billion and hired 3.8 million people (J.S. Bureau of the Census 2010a,2010b).

Even if the population of interest for a survey is a small town of 20,000 or students in a university of 10,000, researchers will have to sample. The costs of surveying thousands of

Census:

?,9* r,j "r.r {:it i * *hi r*l i rit rs r mati rs * i * rsltt *.i * * ti llir*ufilt tlt* t**{}*"ts*fr thel all t v aiia*i * rfi *{n1;} {i r *',Ji ?itl *ntir* pfipul*.tirtrr Srv* tt {l uosli*i}s,

CHAPTER 5 o SAMPLING 123

Exhibit 5.3 Representative and Unrepresentative Samples

Probability sampling

methods:

$ampiin$ rn*thrrls tha.t rcl,,l

*n a ran**m, *r *han**, ;*e lertir:n nrtl:o* *o that th* i:r*h*hility rut **l*r,tirs* r{ prpui*tirs* *icr*e rit* is k**wr,,

Nonprohabi lity sampling methods:

$ainpli*S rrteih*d* in ivhich

the pr*hahility rf s*l**li** *f F:*r:ui*ti** ei*ments is u n krrolvn ,

Probability of selection:

Th* iike lih**d that an rlemtlnt'*riil hc se le*t*d f y1rffi lhe fofulati*n f*r in*irrsifin inlh* sampl*, ln a fi*r]${J$ at all thr rlrrfi*nts of a popu latirs*, th* pr *bab i I i t'1l

that any frartitnsiar el*nr*nt

',,vill hs s*l*r:terl is l,*, ls*r:a.u*e *u*ryono v'rill b*

se l**ted, It half the *i*me nts

i* ihe pr:prJlalirn vrill i:t sa.nrpled on th* bi.rsis r:f

*hanq:c {**y,lsU t*ssiil$ a *c i rr ), i.h*,pr tsbal:ilitv *l s*l**ii*n frlr *a*h *l*m*nt is *** halt, r r , $, V,lh** tha *iru of th* de sirsd sarnple as a

fir*f:*rti*n *l lb,* fr*flulalittn de rrea*e*, s* ches ihs prchahi I it'l *f s*l*rtion,

Random selection:

The funclarne ntal *l*m*ni *t fir*bahility sarripks; the ess * nt i a I r:;har a*t*ri sl i c

*t ranf,rsn selet:tirn is that rvrr,1 elrrttrnf *l th* poprrlation l"ias a known and

ind*p*nd r:'*t *ha**e *f he irrfi se lertrd intn thr sarni:h,

individuals far exceed the budgets for most research projects. In fact, even the U.S. Bureau of the Census cannot afford to have everyone answer all the questions that should be covered in the census. So, it draws a sample. Every household must complete a short version of the census (it had seven questions in 2000), and a sample consisting of one in six households must complete a long form (with 53 additional questions) @osenbaum 2000).

Another cosdy fact is that it is hard to get people to complete a survey. Even the U.S. Bureau of the Census (1999) must make multiple efforts to increase the rate of response in spite of the federal law requiring all citizens to complete their census questionnaire. After spending $167 million on publicity @orero 2000), the Bureau still planned up to six aftempts to contact each household that did not respond by mail (J.S. Bureau of the Census 2000).

In most situations, then, it is much better to select a representative sample from the total population so that there are more resources for follow-up procedures that can overcome reluctance or indifference about participation.

SAMPLING METHODS

fu you can probably guess, the most important feature to know about a sample is whether it is truly representative of the population from which it was selected. Sampling methods that allow us to know in advance how likely it is that any element of a population will be selected for the sample are probability sampling methods. Sampling methods that do not reveal the likelihood of selection in advance are nonprobability sampling methods.

Probability sampling methods rely on a random selection procedure. In principle, this is the same as flipping a coin to decide which person wins and which one loses. Heads and ails are equally likely to turn up in a coin toss, so both persons have an equal chance to win. That chance, or the probability of selectiotr, is 1 out of 2, or .5.

Flipping a coin is a fak way to select one of two people, because the selection process harbors no systematic bias. You might win or lose the coin toss, but you lnow that the out- come \ras due simply to chance, not to bias (unless your opponent tossed a two-headed coin!). For the same reason, rolling a six-sided die is afair way to choose one of six possible outcomes (the odds of selection are 1 out of 6, or .17). Dealing out a hand after shuffling a deck of cards is a fair way to allocate sets of cards in a poker game. (The odds of each person getting a particular outcome, such as a frrll house or a flush, are the same.) Simiiarly, state lotteries use a random process to select winning numbers. Thus, the odds of winning a lottery-the prob- ability of selection-are known, even though they are very small (perhaps 1 out of 1 million) compared to the odds of winning a coin toss. As you can see, the fundamental strategy in probability sampling is the random selection of elements into the sample. When a sample is randomly selected from the population, every element has a lnown and independent chance of being selected into the sample.

There is a natural tendenry to confuse the concept of probability, in which cases are selected only on the basis of chance, with a haphazard method of sampling. On first impres- sion, leaving things up to chance seems to imply the absence of control over the sampling method. But to ensure that nothing but chance influences the selection of cases, the researcher must actually proceed very methodically and leave nothing to chance except the selection of the cases themselves. The resedrcher must carefrrlly follow controlled procedures if a purely random process is to occur. In fact, when reading about sampling methods, do not assume that a random sample was obtained merely because the researcher used a random selection method at some point in the sampling process. Look for these two particular problems: select- ing elemens from an incomplete list of the total population and failing to obtain an adequate response rate (say, only 45% of the people who were asked to participate actually agreed).

If tJne samplingframe-the list from which the elements of the population are selected- is incomplete, a sample selected randomly from the list will not be random. IIow can it be

124 SECTION I1 . FUNDAMENTALS OF RESEARCH

when the sampling frame fails to include every element in the population? Even for a simple population such as a university's student body, the registrar's list is likely to be at least a bit out of date at any given time. For example, some students will have dropped out, but their status will not yet be officially recorded. Although you may judge the amount of error introduced in this particular situation to be negligible, the problems are gready compounded for a larger population. The sampling frame for a city, state, or nation is always likely to be incomplete because of constant migration into and out of the area. Even unavoidable omissions from the sampling frame can bias a sample against particular groups within the population.

An inclusive sampling frame may still yield systematic bias if many sample members can- not be contacted or refuse to participate. Nonresponse is a major hazard in survey research, because individuals who do not respond to a srrvey are likely to differ systematically from those who take the time to participate. You should not assume that findings from a randomly selected sample will be generalizable to the population from which the sample was selected if the rate of nonresponse is considerable (certainly not if it is much above 30%).

Probability Sampling Methods

Probability sampling methods are those in which the probability of selection is known and is not zero (so there is some chance of selectiag each element). These methods randomly select elements and therefore have no systematic bias; nothing but chance determines which elements are included in the sample. This feature of probability samples makes them much more desirable than nonprobabiJity samples when the goal is to generalize to a larger population.

Nonresponse:

W*nnl* nr *,fh*t' nntiiin+ r,rihn X \.rt.ly1\J \.Jt \.Jl.l flrl 1r,lt.l!.,!J() C{}111

iln *rn! n';rii,rirt:rlp irr r tJridriut tt\) i yut rluttJLLtl) r1t e vtutl *lllt*ugh +,h*y a{* fi*l+ct*d ,-.. il- - ^^.*^.^1^1 t \{ t \ 1t} \:,_tf\ 1t1\ /.} t tJt I l1\J L)t-)l l1\J 1Lr,

Systematic bias:

{} u *t r r *p r *fr {} {i,ali * n r: r

u{\!ti { *f}r ***nteti {}{t *t fi*ffi* fl t\ {'1 t tl,s I i re tt {, lt,^t f ";ir I * i" i ti i r,,* t rt l.)\-11:)\ltLtLl\J1 t tfl,ttt t/,\J1,1-'1 lutJUU li,

n idr<1*ln unn',1ii..*, $nn."n f,1"".'{ il1t:r11ttu I u!_:Jt.;11t7 l11Jtl1 l1\U 1.t.,..!t.i.tl,iltu I (r\)Lllr:1l\j tt'*tl i] t:]1,

rn*il,,nfi ir**r'l in *alciri th* l:tL, ltllJLl \-li1-r\^1 l.Lj {-}lJ!!JL,L {i{L.

*?J"t\1{}i*" a *;tntssl* slia**{i l},f ^r,*i".o,*f

i n r'r,? rtli*,* ",vrr,r

ir, &?+t#ill#llrr,){il{ifJ1ltlt-l 131 ! Ui l)

't tnirrr+xil +*rr:nla

LL tJ 1 t.t,\-j \.t \-l i)qAt I I lJ I LJ I

CHAPTERs o SAMPLTNG 125

Even though a random sample has no systematic bias, it certainlywill have some sampling error due to chance.The probability ofselecting heads is .5 il a single toss of a coin-and also in 20, 30, or however manytosses of a coin you like. Be aware, howeveq that itis perfecdypossible to toss a coin twice and get heads both times. The random sample of the two sides of the coin is selected in an unbiased fashion, but it still is unrepresentative. Imagine randomly selecting a sample of 10 people from a population comprising 50 men and 50 women.Just by chance, it is possible that your sample of 10 people will include 7 women and only 3 men. Fornrnately, we can determine mathematically the likely degree of sampling error in an estimate based on a random sample (as you will see later in this chapter), assuming that the sample's randomness has not been destroyed by a high rate ofnonresponse or by poor control over the selection process.

In general, both the size of the sample and the homogeneity (sameness) of the population affect the degree of error due to chance; the proportion of the population that the sample represents does not. Jb elaborate:

The Larger the Sample, the More Confidence We Can Have in the Sample's Representatiyeness of the Population From Which lt Was Drawn. If we randomly pick five people to represent the entire population of our city, our sample is unlikely to be very representative of the entire population in terms of age, gender, race, attitudes, and so on. But if we randomly pick 100 people, the odds of having a representative sample are much better; with a random sample of 1,000, the odds become very good indeed.

The More Homogeneous the Population, the More Confidence We Can Have in the Representatiyeness of a Sample of Any Particular Size. Let us say we plan to draw samples of 50 from each of wo communities to estimate mean family income. One community is very diverse, with family incomes ranging from $12,000 to $85,000. In the more homogeneous community, family incomes are concentrated in a narrower range, from $41,000 to $64,000. The estimated average family income based on the sample from the homogeneous community is more likely to be representative than is the estimate based on the sample from the more heterogeneous community. With less variation, fewer cases are needed to represent the larger population.

The Fraction of theTotal Population That a Sample Contains Does Not Affect the Sample's Representatiyeness, UnlessThatFractionlsLarge, Wecanregardanysamplingfractionunder 2% .vrrth about the same degree of confidence (Sudman 1976).Iafact,sample representativeness is not likely to increase much until the sampling fraction is quite a bit higher. Other things being equal, a sample of 1,000 from a population of I million (with a sampling fraction of 0.001 or 0.1%) is much better than a sample of 100 from a population of 10,000 (although the samplingfractionis0.0l or 1%,whichis l0timeshigher).The sizeof asampleiswhatmakes representativeness more likely, not the proportion of the whole that the sample represents.

Polls that predict presidential election outcomes illustrate both the value of random sam- pling and the problems that it cannot overcome. In most presidential elections, pollsters have accurately predicted the outcomes of the actual vote by using random sampling and, these days, phone interviewing to learn which candidate voters intend to choose. Exhibit 5.4 shows how close these sample-based predictions have been in the last 13 elections. The big excep- tion was the 1980 election, when a third-party candidate had an unpredicted effect. Other- wise, the small discrepancies between the votes predicted through random sampling and the actual votes can be attributed to random error. In 2008, the final Gallup prediction of 55% for Obama was within 2 percentage points of his winning total of 53%. Because they do not disproportionately select particular groups within the population, random samples that are successfirlly implemented avoid systematic bias. In the most recent presidential election, Hill- ary Clinton was predicted to win the popular vote, and although she did win that, in the last

SECTION II . FUNDAMENTALS OF RESEARCH126

Presidential Elections, Predicted Poll and Vote, 1 956-201 6

80

0 1 956 1980 1988

Year

1 996 2012 201 6

Predicted *-w** Result

Source: Gallup. 2011. "Election Polls-Accuracy Record in Presidential Elections;" Panagopoulos, Costas. 2008. "Poll Accuracy in the 2008 Presidential Election;" Jeffrey Jones, November 9,2O72,"Gender Gap in 2012 Vote Is Largest in Gallup's History;" Obama's Road to the White House: A Gallup Review Race was tight until convention period followed by economic crisis," November 5, 2008.

days before the election, undecided voters in so-called batdeground states broke for Donald Tiump. Moreover, on election day, Tiump supporters ftrn out in higher than predicted rates (Cohn 2017). Because they do not disproportionately exclude or include particular groups within the population, random samples that are implemented successfully avoid systematic bias in the selection process. The four most common methods for drawing random samples are (1) simple random sampling, (2) systematic random sampling, (3) stratified random sam- pling, and (4) multistage cluster sampling.

Simple Random Sampling

Simple random sampling requires a procedure that generates numbers or identifies cases stricdy on the basis of chance. fu you know, flipping a coin and rolling a die can be used to identi{, cases strictly on the basis of chance, but these procedures are not very efficient tools for drawing a sample. A random number table, which can be obtained from many websites, simplifies the process considerably. The researcher numbers all the elements in the sampling frame and then uses a systematic procedure for picking corresponding num- bers from the random number table. (Exercise 2 at the end of the chapter explains the pro- cess step by step.) Nternatively, a researcher rnay use a lottery procedure. Each case number is written on a small card, and then the cards are mi*ed up and the sample is selected from the cards.

When a large sample must be generated, these procedures are very cumbersome. Fortu- nately, a computer program can easily generate a random sample of any size. The researcher must first number all the elements to be sampled (the sampling frame) and then run the com- puter program to generate a random selection of the numbers within the desired range. The elements represented by these numbers are the sample.

Organizations that conduct phone surveys often draw random samples with another automated procedure called random digit diding @DD). A machine dials random numbers

L o E60

=L.o ;40 E) cl+t F

820 L o o.

Simple random sampling: 'k

nr*lh*d rtf ii**srSlirrg in \t i lti t. ri i4\i i,. i \ t <t 2 r {1. i}1. * *l *ff I * f i trir,:iLri, ...1 t\:J

| ,J<.tIii,

i* n*ir:t;le* *fi1\i {}fr l.h* bri'r,it; *i ti t;l {i t,* illi *u Uit'rt { ?trtd*rit ?)1 TJtL\J\)\].

Bandom number tahle: 'A t;:,*l* *rsrsl'*-inirz;i iitir.* t:li

?,ttTi**{5 lit'dt iJ {'r *r .1*r *,ri tj{:ixl,i {s{\ tliF: n?sit ,:t ihnr,*, ii ;." ,,,rr,r'l {r,n ,1u,,li iii 'Jl*u ii-ll f,ll nwi{i{} ;" r r:.n rJ ryrt gairtlll *,

Handom digit dialing (RI)D): iii* r;:,n{l{}ni iltalirt7lsi it

ry1ai:rti rt* rrt nu{rtb{rr * witl*n

d cg i tJtlait,d frli * r * fi | {:fty, QS, wlii r:h t) { * ?.tT** it t {*i {::i {tty1

t:itfftplr: i'*r vttl*ts* i;tJr, iF:T1,

CHAPTER 5 . SAMPLING 127

Exhibit 5.4 Election Outcomes: Predicted and Actual

Beplacement sampling:

A m*thi:tj *t ,ranspling in v'ihich saffiillc *l*m*ntg ar* r*ti.trrtad irs th* ,*arnfrlinfr

tram* att*r h*ing scl**t*r! *rs t*qu mau bt, sar*pl*d

*#zti*, fra*tl*tr sam p i * * ni a'l b* **lr:*\ed '+*ith *r wiilsrs*r r*'rtla*r,rri**i..

within the phone prefixes corresponding to the area in which the survey is to be conducted. RDD is particularly useful when a sampling frame is not available. The researcher simply replaces any inappropriate numbers (e.g., those no longer in service or for businesses) with the next randomly generated phone number.

As the fraction of the population that has only cell phones has increased (40o/" in 2013), it has become essential to explicidy sample cell phone numbers as well as landline phone numbers (McGeeney and Keeter 2014). Those who use only cell phones tend to be younger, male, and single and are more likely to be black or Hispanic and less likely to vote compared with those who have a landline phone. fu a result, failing to include cell phone numbers in a phone survey can introduce bias (Christian, Keeter, Purcel, and Smith 2010). In fact, in a 2008 presidential election survey, those who used only cell phones were less likely to be registered voters than were landline users but were considerably more favorable to Obama than landline users (Keeter 2008).

In the National Intimate Parmer and SexualViolence Survey (I\ISVS) conducted by the Centers for Disease Control and Prevention (CDC), both landline and cell phone databases of adult U.S. residents were selected through an RDD random sampling method @lack et al. 2011). You will learn more about this survey in Chapter 8.

The probability of selection in a true simple random sample is equal for each element. If a sample of 500 is selected from a population of 17,000 (i.e., a sampling frame of 17,000), then the probability of selection for each element is 500 out of 17,000, or .03. Every element has an equal and independent chance ofbeing selected; these odds are the same as the odds in a toss of a coin (1 in 2) or a roll of a die (1 in 6). Thus, simple random sampling is an equal probability of selection metbod (EPSEM).

Simple random sampling can be done either with or without replacement sampling. In replacement sampling, each element is returned to the sampling frame from which it is selected so that it may be sampled again. In sampling without replacement, each element selected for the sample is then excluded from the sampling frame. In practice, it makes no difference whether sampled elements are replaced after selection, as long as the population is large and the sample is to contain only a small fraction of the population.

For the CDC's MSVS study, noninstitutionalized (e.g., not in nursing homes, prison, and so on) English- and/or Spanish-speaking residents aged 18 and older were randomly selected through an RDD sampling method in 2010. A total of 9,970 women and 8,079 men were selected. Approximately 45o/" of the interviews were conducted by landline and 55% by cell phone. The final sample represented the U.S. population very well. For example, the proportion of the sample by gender, race./ethnicity and age in the MSVS sample was very close to the sample proportions for the U.S. population as a whole.

How does this sample strike you? Let us assess sample quality usiag the questions posed earlier in the chapter:

o Fronr. wbat population were the cases selected.? There is a clearly defined population: the adult residents of the continental United States (who live in households with phones).

c Wat method usas used to select cases from this populationi The case selection method is a random selection procedure, and there are no systematic biases in the sampling.

o Do the cases that were studied represent, in the aggregate, the pypalation from wbich tbey were selected?The findings are very likely to represent the population sampled, because there were no biases in the sampling and a very large number of cases were selected. However, it must be remembered that an average of 30% of those selected for interviews could not be contacted or chose not to respond. This rate of nonresponse may have created a small bias in the sample for several characteristics.

128 sECTroN n . FUNDAMENTALS oF RESEARCH

We also must consider the issue of cross-population generalizability. Do findings from this sample have implications for any larger group beyond the population from which the sample was selected? Because a representative sample of the entire U.S. adult population was drawn, this question has to do with cross-national generalizations.

Syslematic Random Sampling

Systematic random sampling is a variant of simple random sampling and is a litde less time consuming. When you systematically select a random sample, the first element is selected randomly from a list or from sequential files and then every zth element is systematically selected thereafter. This is a convenient method for drawing a random sample when the population elements are arranged sequentially. It is particularly efficient when the elements are not actually printed (i.e., there is no sampling frame) but instead are represented by folders in fiIing cabinets.

Systematic random sampling requires three steps:

1. The total number of cases in the population is divided by the number of cases required for the sample. This division yields the sampling interval, the number of cases from one sampled case to another. If 50 cases are to be selected out of 1,000, the sampling interval is 20 (1,000 / 50 = 20); every 20th case is selected.

2. A number between I and 20 (the sampling interval) is selected randomly. This number identifies the first case to be sampled, counting from the first case on the Iist or in the files.

3. After the first case is selected, every nth case is selected for the sample, where z is the sampling interval. If the sampling interval is not a whole number, the size of the sampling interval is systematicallyvaried to yield the proper number of cases for the sample. For example, if the sampling interval is 30.5, the sampling interval alternates between 30 and 3 1

In almost all sampling situations, systematic random sampling yields what is essentially a simple random sample. The exception is a situation in which the sequence of elements is affected by periodicity; that is, the sequence varies in some regular, periodic pattern. The list or folder device from which the elements are selected must be truly random in order to avoid sam- pling bias. For example, we could not have a list of convicted felons sorted by offense q'pe, age, or some other characteristic of the population. If the list is sorted in any meaningful way, this will introduce bias to the sampling process, and the resulting sample is not likely to be represen- tative of the population. But in reality, periodicity and the sampling interval are rarely the same.

Stratifi ed Random Sampling

Although all probability sampling methods use random sampling, some add steps to the sam- pling process to make sampling more efficient or easier. Samples are easier to collect when they require less time, money, or prior information.

Stratified random sampling uses information known about the total population prior to sampling to make the sampling process more efficient. First, all elements in the popula- tion (i.e., in the sampling frame) are differentiated on the basis of their value on some rel- evant characteristic. This sorting step forms the sampling strata. Next, elements are sampled randomly from within these strata. For example, race may be the basis for distinguishing individuals in some population of interest. Within each racial category selected for the strata, individuals are then sampled randomly.

Why is this method more efficient than drawing a simple random sample? Well, imagine that you plan to draw a sample of 500 from an ethnically diverse neighborhood. The neigh- borhood population is 15% A-frican Ameri carr,l0o/o Hispanic, 5% fuian, and70%" Caucasian.

Systematic random

sampling:

h m*+ift*rj *f *anfr\in U i n vrtlj *'sanfrl* **xyt**t* ar *, sxi*ctec tr*n ali*+, cr f rorn Ecqu*ntial filcs, ,,rilh *v*r,i rftli *l*m*nt ls*in* s * I * cterj 'aft*r ihe first *l*m*n,is selcrt*ri randrsnlv u,,ithin thc ttrr*t intrrva!,

SamplinU interval:

Th* nLrnrhrr r:t caso$ b*twc*l'i *fi8 srrixpl*rl rase and thc ri** ifi a s',,$tsln*tir ralrd*m *anplx,

Periodicity:

A s*queir** af *l*m*nts

tin alist tr: br *arrrpled) tliatv"arics iti $ot;t* r*gt:\ar, p*rirrii* patt*rn,

Stratified random sampling:

A ryr*th*d *f samplinC in vrhich sarfipie eler*e nls ar* Exl**t*d sep;zrately f r*m p o p Lr lati *n *tr *ta lli*t *r * i d e r: lifi *ri irt atj'; a*** Ys,; ilt* r***art?t*r,

CHAPTER 5 ' SAMPLlNG 129

Proportionate stratifi ed

sampling:

$airrpli*# m*thrsd* ut vrhich

strata i* *xa*t i:r"tp*rtirin t* ih*ir repr*$8lrtalir:n ifi tlts

f,]*[iLJlatir:n,

Disproportionate stratifi ed

sampling:

$arnpling in l,uhich rl*m*nts are s*lr*t*rJ tr*m xtrni.a in ditl*r**i" p r*r:* rti * ns f ro*'r

thr:s* that a{:ts**r in lk* fr{}'frt} lati*n,

If you drew a simple random sample, you might end up with disproportionate numbers of each group. But ifyou created sampling strata based on race and ethnicity, you could ran- domly select cases from each stratum: 75 AfricanAmericans (15% of the sample), 50 Hispan- ics (10%), 25 Asians (5"/"), and 350 Caucasians (70o/"). By using proportionate stratified sampling, you would eliminate any possibility of error in the sample's distribution of etlnic- ity. Each stratum would be represented exacdy in proportion to its size in the population from which the sample was drawn (see Exhibit 5.5).

In disproportionate sffatified sampling, the proportion of each stratum thatis included in the sarnple is intentionally varied from what it is in the population. In the case of the sample stratified by ethnicity, you might select equal numbers of cases from each racial or ethnic group: l25AfricanAmericans(25o/o of thesample), 125 Hispanics(25o/"),125Asians QSo/"), and 125 Caucasians (25y.).In this type of sample, the probability of selection of every case is known but unequal between strata. You know what the proportions are in the population, so you can easily adjust your combined sample accordingly. For instance, if you want to combine the ethnic groups and estimate the average income of the total population, you would have to weight each case in the sample.The weight is a number you multiply by the value of each case based on the stratum it is in. For example, you would multiply the incomes of all African Americansinthesampleby0.6(75 / 125),rheincomesofallHispanicsby0.4 (50/ 125),and so on. Weighti"g i" this way reduces the influence of the oversampled strata and increases the influence of the undersampled strata to what they would have been if pure probability sampling had been used.

Population: All residents of community X t7 = 10,000

Asian n=500

5"/o Random selection:

1 in 56 from White stratum; 1 in 8 from Hispanic stratum; 1 in 1 2 trom Black stratum;

1 in 4 from Asian stratum

\

Random selection: 1 in 20 from each stratum

Black ll,= 75 15%

Hispanic /?=50 10% n

ll :25 5%Proportionate sample,

n=500 Disproportionate sample,

n=500

130 SECTION II . FUNDAMENTALS OF RESEARCH

Exhibit 5.5 Stratified Random Sampling

Why would anyone select a sample that is so unrepresentative in the first place? The most common reason is to ensure that cases from smaller strata are included in the sample in suffi- cient numbers. Only then can separate statistical estimates and comparisons be made between strata (e.g., between African Americans and Caucasians). Remember that one determinant of sample quality is sample size. The same is true for subgroups within samples. If a key concern in a research project is to describe and compare the incomes of people from different racial and ethnic groups, then it is important that the researchers base the mean income of each group on enough cases to be a valid representation. If few members of a particular minority group are in the population, they need to be oversampled. Such disproportionate sampling may also result in a more efficient sampling design if the costs of data collection differ mark- edly between strata or if the variability (heterogeneiry) of the strata differs.

Multistage Cluster Sampling

Although stratified sampling requires more information than usual prior to sampling (about the size of strata in the population), multistage cluster sampling requires less prior infor- mation. Specifically, cluster sampling can be useful when a sampling frame is not available, as often is the case for large populations spread across a wide geographic area or among many different organizations. In fact, if we wanted to obtain a sample &om the entire U.S. popula- tion, there would be no list available. Yes, there are lists in telephone books of residents in various places who have telephones, lists ofthose who have registered to vote, lists ofthose who hold driver's licenses, and so on. However, all these lists are incomplete; some people do not list their phone number or do not have a telephone, some people are not registered to vote, and so on. Using incomplete lists such as these would introduce selection bias into our sample.

In such cases, the sampling procedures become a litde more complex, and we usually end up working toward the sample we want through a series of steps or stages (rence the namemahistagel). First, researchers extract a random sample of groups or clusters of elements that are available, and then they randomly sample the individual elements of interest from within these selected clusters. So, what is t claster? A cluster is a naturally occurring, mixed aggregate of elements of the population, with each element appearing in one and only one cluster. Schools could serve as clusters for sampling students, blocks could serve as clusters for sampling city residents, counties could serve as clusters for sampling the general population, and businesses could serve as clusters for sampling employees.

Drawing a cluster sample is at least a two-stage procedure. First, the researcher draws a random sample of clusters. A list of clusters should be much easier to obtain than a list of all the individuals in each cluster in the population. Next, the researcher draws a random sample of elements within each selected cluster. Because only a fraction of the total clusters is involved, obtaining the sampling frame at this stage should be much easier.

In a cluster sample of city residents, for example, blocks could be the first-stage clusters. A research assistant could walk around each selected block and record the addresses of all occupied dwelling units. Or in a cluster sample of students, a researcher could contact the schools selected in the first stage and make arrangements with the registrars to obain lists of students at each school. Exhibit 5.6 displays the multiple stages of a cluster sample.

How many clusters and how many individuals within clusters should be selected? As a general rule, cases in the sample will be closer to the true population value if the researcher maximizes the number of clusters selected and minimizes the number of individuals within each cluster. Unfortunately, this srategy also maximizes the cost of the sample. The more clusters selected, the higher the travel costs. It also is important to take into account the homogeneity of the individuals within clustersl the more homogeneous the clusters, the fewer cases needed per cluster. This should make intuitive sense, as it is more likely that any selected element will represent other elements if they are alike. Although cluster sampling is a very

Multistage cluster sampling: ,\ am*lirr u i n,tr fii *h *i:n * rfi * ar* **l**t*tl i* fiurs *t rfiNo *tag*s, uuir"hth* flrst slrl#e

*rti',,tg lh* randrs'xt **l**ti*n *t n'dlura I iy i:mivrinfr +iritt*rr *n(t +,h* ln*t r,tag* ls*tnrs th* ra*d*rn scl*cti*n rst rntsl ii I eve I * kn:* rlts wii.ltiti ,;lsr,t*rx,

Cluster:

h'ff?tt tlr nll y rs * r;*r rirtl,, rnix*d

afrgr*frat* *t *l*r,**t* *t tts*, itilplJl*ti*n,

131CHAPTER 5 O SAMPLING

Stage 4: Randomly select students within

each school

popular method among survey researchers, it has one drawback Sampling error is greater in a cluster sample than in a simple random sample. And as a general rule, this eror increases as the number of clusters decreases.

Many federal government-funded surveys use multistage cluster samples or even combi- nations of cluster and stratified probability sampling methods. For example, the U.S. Justice Departrnent's National Crime Victimizadon Survey O{CVS) is an excellent example of a multistage cluster sample. In the NCVS, the first stage of clusters selected are referred to as priruary sampling units (PSUs), and they represent a sample of rural counties and large metropolitan areas. From these PSUs, the second stage of sampling involves the selection of geographic districts within each of the PSUs that have been listed by the U.S. Bureau of the Census population census. Finally, a probability sample of residential dwelling units is selected from these geographic districts. These dwelling units, or addresses, represent the final stage of the multistage sampling. Anyone who resides at a selected address who is 12 years of age or older and is a U.S. citizen is eligible for the NCVS sample. Approximately 50,500 housing units or other living quarters are designated for the NCVS each year and are selected in this manner.

:"-;;:,:"0::o:;:;::::,":,^:;';;;:L--:;::ffi ::",':.T:. each cluster.

What metboilwas wed to select casesfrom tbis population?The random selection method was carefully described.

Do tbe cases that were stadied. represent, in the aggregate, the population from wbich thelt zlere selexedi The unbiased selection procedures make us reasonably confident in the representativeness of the sample.

Nonprobability Sampling Methods Unlike probability samples, nonprobability samples comprise elements within a population that do not have a known probability of being selected into the sample. Thus, because tJre chance of any element being selected is unknown, we cannot be cerain the selected sample actually represents our population. Why, you may be asking yourself right now, would we want to use such a sample if we cannot generalize our results to a larger population? These methods are useful for several pu{poses, including those situations in which we do not have a population list, when we are exploring a research question that does not concern a large

SECTION II . FUNDAMENTALS OF RESEARCH

Stage 1: Randomly

select states

Stage 2: Randomly select

cities, towns, and counties within those states

Stage 3: Randomly select

schools within those cities and

towns

132

Exhibit 5.6 Multistage Gluster Sampling

population, or when we are doing a preliminary or exploratory study. Nonprobability sam- pling methods are often used in qualitative research, when the focus is on one setting, or when a small sample allows a more intensive portrait of activities and actors. Suppose, for example, that we were interested in the crime of shoplifting and wanted to investigate how shoplifters rationalize their behavior. For example, do they think insurance covers the prod- ucts they steal, so they are not really hurting anyone? It would be hard to define a population of shoplifters in this case, because we do not have a list of shoplifters from which to randomly select. There may be lists of convicted shoplifters, but of course, they represent only those shoplifters who were actually caught. fu you can see, producing a random sample of some groups is very difEcult indeed.

There are four common nonprobability sampling methods: (1) availability sampling, (2) quota sampling, (3) purposive or judgment sampling, and (4) snowball sampling. Because these methods do not use a random selection procedure, we cannot expect a sample selected with any of these methods to yield a representative sample. They should not be used in quantitative studies if a probability-based method is feasible. Nonetheless, these methods are useful when random sampling is not possible, when a research question calls for an intensive investigation of a small population, or when a researcher is performing a preliminary exploratory study.

Availability Sampling

In availability sampling, elements are selected because they are available or easy to find. Con- sequendy, this sampling method is also known as-baphazaril, accidmtal, or conaenience sampling. fu noted earlier, news reporters often use person-on-the-street interviews, a qpe of availabil- ity sample, to inject a personal perspective into a news story and show what ordinary people may think of a given topic. Availability samples are also used by university professors and researchers all the time. Have you ever been asked to complete a questionnaire before leaving one of your classes? If so, you may have been selected for inclusion in an availability sample.

Even though they are not generalizable, availability samples are often appropriate in research-for example, when a field researcher is exploring a new setting and trying to get some sense of prevailing attitudes or when a survey researcher conducts a preliminary test of a questionnaire. There are a variety of ways to select elements for an availability sample: stand- ing on street corners and talking to anyone walking by; asking questions of employees who come to pick up their paychecks at a personnel office; or distributing questionnaires to an available and captive audience, such as a class or a group at a meeting. Availability samples are also frequendy used in fieldwork studies interested in obtaining detailed information about a particular group.

One classic example of an availability sample used in research is found in John Irwin's Tbe Felon (1970), a snrdy of the postprison careers of a group of California criminal felons. To conduct his study, Irwin did not take a simple or systematic random sample of all felons released from the California Departrnent of Corrections and Rehabilitation. Instead, because he resided and worked in the San Francisco area, Irwin studied offenders released on parole from penal institutions in San Francisco and Oakland parole districts who volunteered for his study.

When such samples are used, it is necessary to explicidy describe the sampling proce- dures used in the methodology section of research reports to acknowledge the nonrepre- sentativeness of the sample. For example, in a study investigating the prevalence of problem behavior in a sample of students pursuing policing careers, Gray (2011) stated,

a convenience/purposive sample was used to survey studenB attending a medium-sized public, Midwestern university . . . to determine if differences existed between students majoring in criminal justice (C) and students with other majors in terms of deviance and delinquency, drinking and drug use, and an array of other behaviors. (544)

Availability sampling: !;n r,sriii r: * i n t,t'l ii tll: t: i t::tt1* tl\S\,)tAl t lt.J i l tll_j til tt l i ti_ n uo .,o l^l.r,til itn il',r, \+oi,:. ,r{ 'JJO.j*iU1", lUl.r l,Jll Lllry 1"!L13lb Ul

t llri\lultiurll'u \./ t.j 1 ] i; \.! | 1 i \.J 1 t \J \..! 1

CHAPTER 5 SAMPLING 133

0uota sampling:

A n*n i:r*bahi I ity*ar*frltn*

m*l*tsd in tluh ir:lt *l*rytr;*s ar * s*l**!*d tts o n$il r"s l*at th* sam p i * r *{sr**rt*!* *ertai* c hiira*k ri sti ils ir, pr*p*rti** t* th*ir prevaitnco i* ihc $*flulatirit,

Those ofyou studying for a policing career will be interested to know that over one quarter of the CJ majors had engaged in serious forms of problematic behavior, such as marijuana use. Students who engage in these forms of illegal activities, Gray points out, "should expect to have some level of difficultywith police application and hiring processes" (2011, 549). Butwe digress.

IIow does this sample compare to a simple random sample? There is no clearly definable population from which the respondents were drawn, and no systematic technique was used to select the respondents. ConsequendSz, there is not much likelihood that the sample is repre- sentative of any target population; the problem is that we can never be sure.

Availability sampling often masquerades as a more rigorous form of research. Popular magazines and Internet sites periodically survey their readers by asking them to fill out questionnaires. Follow-up articles then appear in the magazine or on the site, displaying the results under such titles as "What You Think About the Death Penalty for Teenag- ers." If the magazinet circulation is large, a large sample can be achieved in this way. The problem is that usually only a tiny fraction ofreaders fill out the questionnaire, and these respondents are probably unlike other readers who did not have the interest or time to participate. So, the survey is based on an availability sample. Polls conducted on specific websites (not online surveys, which we will discuss later) are also typically availability samples. Have you ever quickly answered the question of the day on the CNN or ESPN websites? These samples are suspect not only because they are availability samples, but also because electronic surveys remain very susceptible to electronic ballot stuffing; few sites restrict access, so you can answer the posted questions as many times as you want. Because of the nonscientific basis of these polls, many Internet sites that conduct such polls, including ABC.com, now add this disclaimer to the online poll's question of the day: "Not a scientific poll; for entertainment only."

Quota Sampling

Quota sampling is intended to overcome availability sampling's biggest downfall: the likeli- hood that the sample will only consist of who or what is available, without any concern for its similarity to the population of interest. The distinguishing feature of a quota sample is that quotas are set to ensure that the sample represents certain characteristics in proportion to their prevalence in the population.

Quota samples are similar to stratified probability samples, but they are generally less rigorous and precise in their selection procedures. Quota sampling simply involves divid- ing the population into proportions of some groups that you want to be represented in your sample. Similar to stratified samples, in some cases, these proportions may actually represent the true proportions observed in the population. At other times, these quotas may represent predetermined proportions of subsets of people you deliberately want to oversample.

Suppose we are interested in investigating the crime of personal larceny with contact, which generally involves pocket picking and purse snatching. Because elderly citizens are disproportionately victimized by this crime, we would want to make sure we have enough elderly victims in our sample to make comparisons to younger cohorts. For a study such as this, we may decide that we want 50% of the victims in our sample to consist of individuals 65 years of age or older and 50o/o to consist of those younger than 65 years of age. When these proportions are decided, they represent o:ur qaotas. Elements of the population are then collected until each quota is filled. In our example, we would select the sample until we have exactly 50% of the elements younger than 65 and 50"/" 65 years of age or older.

Quota sampling may be more complex in that we can add quotas to fill. For example, in addition to age, we may want to make sure that within each age group, rve have a certain proportion of African American men and women, a certain proportion of Hispanic men and women, and a certain proportion of Caucasian men and women.

134 sEcrroN n . FUNDAMENTALS oF RESEARCH

Quota Sample 50% male, 50% female

i.-..i @rjw

-q i,",,,fi'i':,,!,r'!'!,,r:, \ ,:' 'i #,........ikli.;.ii =',,'.,

ll::l::'::::':'::!:,:::\ ,:' 'i

]ff"i ij i" Representative of gender distribution in population, not representative of

race distribution

The problem is that even when we know that a quota sample is representative of the particular characteristics for which quotas have been set, we have no way of knowing if the sample is representative in terms of any other characteristics. Exhibit 5.7 shows an example in which quotas have been set for gender only. Under the circumstances, it's no surprise that the sample is representative of the population only in terms of gender, not in terms of race. Inter- viewers are only human and guided by their own biases; they may avoid potential respondents with menacing dogs in the front yard, or they could seek out respondents who are physically attractive or who look like they would be easy to interview. Realistically, researchers can set quotas for only a small fraction of the characteristics relevant to a study, so a quota sample is really not much better than an availability sample (although following carefirl, consistent procedures for selecting cases within the quota limits always helps).

This last point leads to another limitation of quota sampling: You must know the characteristics of the entire population to set the right quotas. In most cases, researchers know what the population looks like in terms of no more than a few of the characteristics relevant to their concerns, and in some cases, they have no such information on the entire population. Exhibit 5.8 summarizes the differences between quota sampling and stratified random sampling. The key difference, of course, is quota sampling's lack of random selec- tion. Thus, even though quota sampling techniques range from the simple to the rigorous, all quota samples are still considered nonprobability samples because they do not rely on random selection.

Does quota sampling remind you of stratified sampling? It's easy to understand why, because they both select sample members pardy on the basis of one or more key characteris- tics. The key difference is quota sampling's lack of random selection.

Purposive or Judgment Sampling

In purposive sampling each sample element is selected for a purpose, usually because of the unique position of the sample elements. It is sometimes referred to as jud.gmmt sampling, because the researcher uses his or her own judgment about whom to select into the sample rather than drawing sample elements randomly. Purposive sampling may involve studying the

Population 50% male, 50% female

70% white, 30% nonwhite

. ,i-ilii.i "i

i i'""'i.i i it' 'ii

i i ,,'i i

i"ii j i ji i i i"i i,,,i,: i"li i

i..:i".l @@6} riir l'ffiWffi

,; i:

,',, ,

, :.,;' 'l;,;

i'ti;

Purposive samplinU:

F, n * "* ts r *ls'abili'tt tla"ifr llli ft lj

w,tl1{}ii in vyh ith *icm*nls er* tifil**l*rl I*r t- #u{*tt**, t-iilr';tily l:*r;a*s* *! tlt*ir I i {1i riti * fr {} tiili t: rt,,$ * r* *tt {{t * ?, rrtlr:rr*ri ii: as jud#rnsxl ;:'s tnnlin rt \)r.tt!ll.trtttvt

CHAPTER5 o SAMPLING 135

Exhibit 5.7 Quota Sampling

Population

Representative sample

Quota sample, representative only by gender

entire population of some limited group (members of a street gang) or a subset of a popula- tion (uvenile parolees). Or a purposive sample may be a key informant surve)a which targets individuals who are particularly knowledgeable about the issues under investigation.

Rubin and Rubin (1995) suggest three guidelines for selecting informans when design- ing any purposive sampling strategy. Informants should be

. knowledgeable about the cultural arena or situation or experience beingstudied; e willing to trlk; and . representative ofthe range ofpoins ofview.

In addition, Rubin and Rubin (1995) suggest continuing to select interviewees until your sample can pass two tesB:

Completeness. "Whatyou hear provides an overall sense of the meaning of a concept, theme, or process" (2).

Sataraion. "You gain confidence that you are learning little that is new from subsequent interview[s] " (7 3 ).

Adhering to these guidelines will help ensure that a purposive sample adequately repre- sents the setting or issues being studied.

Of course, purposive sampling does not produce a sample that represents some larger population, but it can be exacdy what is needed in a case study of an organization, a commu- nity, or some other clearly defined and relatively limited group. For example, in their book Crirnes of tbe Middle Class,Weisbrrd et al. (1991) examined a sample of white-collar criminal offenders convicted in seven federal judicial districts. These judicial districts were not ran- domly selected from an exhaustive list of all federal districts but were instead deliberately selected by the researchers because the seven districts were judged by them to provide a

SECTION II . FUNDAMENTALS OF RESEARCHr36

Exhibit 5.8 Unrepresentative Quota Sample Versus a Representative Stratifred Sample

suitable amount of geographical diversity.Theywere also selected because theywere believed to have a substantial proportion of white-collar crime cases. The cost of such nonprobability sampling, you should realize by now, is generalizability; we do not know if findings byWeis- burd et al. hold true for white-collar crime in other areas of the country.

Snowball Sampling

Snowball sampling is useful for hard-to-reach or hard-to-identifz populations for which there is no sampling frame but in which the members of which are somewhat interconnected (at least some members of the population lnow each other). Using this technique, you iden- ti[, one member of the population and speak to him or heq and then ask that person to identifir others in the population and speak to them, then ask them to identifi, others, and so on. The sample size increases with time as a snowball would rolling down a slope. This technique is useful for hard-to-reach or hard-to-identi$r interconnected populations where at least some members of the population know each other, such as drug dealers, prostitutes, practicing criminals, gang leaders, and informal organizational leaders.

In their study of juvenile gangs in St. Louis, Missouri, Decker and Van Winkle (1996) utilized the technique of snowball sampling. Specifically, the snowball began with an earlier fieldwork project involving active residential burglars (Wright and Decker 1994). The young members from this sample, along with contacts a field ethnographer had with several active street criminals, started the referral process. The initial interviewees then nominated other gang members as potential interview subjects.

One problem with this technique is that the initial contacts may shape the entire sample and foreclose access to some members of the population of interest. Because Decker and Van Winkle (1996) wanted to interview members from several gangs, they had to restart the snowball sampling procedure many times to gain access to a large number of gangs. One problem, of course, was validating whether individuals claiming to be gang members actually were legitimate members or were so-called wannabes. Over 500 contacts were made before the final sample of 99 was complete.

St.Jean (2007) also used snowball sampling for recruiting offenders in a Chicago neigh- borhood for interviews. After several years of participant observation (see Chapter 9) within a Chicago community, St. Jean wanted to understand the logic offenders used for setting up street drug dealing and staging robberies. He explained his sampling technique as follows:

I was inroduced to the offenders mainly through referrals from relatives, customers, friends, and acquaintances who, after several months (sometimes years), trusted me as someone whose only motive was to understand life in their neighborhood. For instance, the first tlree drug dealers I interviewed were introduced by their close rela- tives. Toward the end of each interview, I asked for leads to other subjects, with the first three interviews resulting in eleven additional leads. (St.Jean 2007 ,26)

More slstematic versions of snowball sampling can also reduce the potential for bias. The most sophisticated version, resp ond.ent-driaen sampling, gSves financial incentives, also calTed gra- tilities,to respondents to recruit peers (Ileckathorn l997).Limitations on the number of incen- tives that any one respondent can receive increase the sample's diversity. Targeted incentives can steer the sample to include specific subgroups. When the sampling is repeated through several waves, with new respondents bringing in more peers, the composition of the sample converges on a more representative mix of characteristics. Exhibit 5.9 shows how the sample spreads out through successive recruitment waves to an increasingly diverse pool (Ileckathorn 1997). fu with all nonprobability sampling techniques, however, researchers using even the most system- atic versions of snowball sampling cannot be confident that their sample is representative of the population of interest.

Snowball sampling:

F, tt*tir*d *f fiarTlijliriU in ivh i c h saniTsl* *l*rrt**ls: nr* **l**1.*r, as l.hrt'v nr*

id*nti{t*d,*,1 ** * * **aiu * i,.^t ^,,,^^.^. - +-. ^.,, l-+ -..,. .: ^... ^ -. -.lJlJttt t t!;ttiL tll !tltuv lt,#1tit6utlll{ (Jl 111t"!.11a\} tJl 111\,\Jl U 1qJ l!\;\rr_},

CHAPTER5T SAMPLTNG 137

./-*ffi# \

\*

wi

6i),

l:::[::::l;::;trlli[:

.lhff',- ,/ -@_' :*ffi>#hE

(89

+[.iiiiiIfiil rl u'9[.iiliil"

m tiEr

l:i::n::!:::!:::ii:!::l

IJtiiiii:lil

lilE:] t:tI::l

UU

Successive waves of sampling gradually produce a more representative sample than is typical of snowball sampling.__ffi

->-F#lnstructions to respondenls: "We'll pay you $5 each for up to three names, but only one of those names can be somebody from your own town. The others have to be from somewhere else."

Lessons About Sample Quality Some lessons are implicit in our evaluations of the samples in this chapter:

We camot evaluate the quality of a sample if we do not know what population it is supposed to represent. Ifthe population is unspecified because the researchers were never clear about what population they were trying to sample, then we can safely conclude that the sample itself is no good.

We cannot evaluate the quality of a sample if we do not know exacdy how cases in the sample were selected from the population. If the method was specified, we then need to know whether cases were selected in a systematic fashion or on the basis of chance. In any case, we know that a haphazard method of sampling (as in person-on-the- street interviews) undermines generalizability.

Sample quality is determined by the sample actually obtained, not only by the sampling method itself. That is, findings are only as generalizable as the sample from which they are drawn. If many of the people (or other elements) selected for our sample do not respond or participate in the study, even though theyhave been selected for the sample, generalizability is compromised.

We need to be aware that even researchers who obainverygood samples maytalk about the implications of their findings for some group that is larger than or different from the population they actually sampled. For example, findings from a representative sample of students in one university often are discussed as if they tell us about university students in general. And maybe they do; the problem is that we dont know.

138 SECTION II . FUNDAMENTALS OF RESEARCH

Exhibit 5.9 Respondent-Driven Sampling-A Version of Snowball Sampling

_@_

>84+ira EIEJ

Generalizability in Qualitative Research

Qualitative research often focuses on populations that are hard to locate or very limited in size. In consequence, nonprobability sampling methods such as availability sampling and snowball sampling are often used. I{owever, this does not mean that generalizability should be ignored in qualitative research, or that a sample should be studied simply because it is convenient (Gobo 2008). Schofield (2002) suggests nvo different ways ofincreasing the gen- eralizability of the samples obuined in such situations:

Stad.yingtbeTypical. Choosingsites on the basis of their fitwith a typical situation is far preferable to choosing on the basis ofconvenience. (181)

CHAPTER 5 o SAMPLING r39

Captain tennifer Griffin, PhD, Patrol TIoop Gommander, Delaware State Police

Captain Jennifer Griffin has been a trooper with the Delaware State Police for 1.6 years. She has had assignments as a patrol trooper, a school resource officer, and a grant man- ager and planner in the Planning Section; she has been the director of

Source: Courtesy of Jennifer the Internal Affairs Unit, Griffin and she is currently a patrol troop commander. But what most people don't know is that she also has a PhD from the University of Delaware. As a young woman, Captain Griffin says she always knew that her career calling was to be a police officer, and so she pursued her bachelor's degree in criminal justice. However, within one year of becom- ing a state trooper, she realized that she wanted to earn her master's degree. During that time, she says, "l realized how much I enjoyed school and research, and how much more the research and the lessons meant to me because I was studying the field in which I was working." After she completed her master's, she con- tinued on and earned her doctorate in sociology with concentrations in gender and deviance.

Captain Griffin's doctoral studies included writ- ing her dissertation, which was titled "stress in the 21st Century: Are We Protecting Those Who Protect Others?" She says she chose a research topic based on her experience as a police officer and wanted to use her research abilities and training to gain infor- mation to help police officers deal with the stress and

strains of policing.Through her research, she was able to open a dialogue between police officers and the Delaware State Police administration to discuss stress, burnout, work-family conflict, and the relationships among officers from different demographic groups.

Captain Griffin's research skills have provided her with opportunities that she says she never even irnagined. As a result of her interest and education, she has been teaching as an adjunct instructor at the University of Delaware and other universities, and she teaches classes that range from graduate-level research methods to several undergraduate criminal justice courses. She says, "Without my PhD, teaching both undergraduate and graduate students wouldn't have been an option, and it gives me the opportu- nity to share not only my expertise as a researcher and scholar, but also my life experiences as a state trooper. Being able to blend *y academic background and life stories really, helps me engage with sturCents in ways that are both meaningful for the students and relevant to what is going on within society."

Her advice to other criminology and criminal jus- tice students is this:

Find an area within the field that you are most interested in, and focus on how you can make it meaningful to not only you but to the field, because you really never know the doors that will open for you in the pro- cess. I could have never imagined all of the opportunities and interests that I have developed just from continuing my studies within criminal justice.

It

Ir_

2a

Sampling distrihution:

fr, ln *, * r *tir: a I d i siril: uLi tw

rsl th* va.l** t:t a. *tatir,rtir; r,,i,*,,m*an) tr*ryt a* n{nilt* nun:he r *l *an*-*it* s*.mpie *,

Perforrning Muhisiu Studies. A finding emerging repeatedly in the study of numerous sites would appear to be more likely to be a good working hypothesis about some as yet unstudied site than a finding emerging from just one or two sites. . . . Generally speaking, a finding emerging from the study of several very heterogeneous sites would be more . . . likely to be useful in understanding various other sites than one emerging from the study of several very similar sites. (184)

Gobo (2008) highlights another approach to improving generalizability in qualitative research. A case may be selected for in-depth study because it is atypical or deviant. Investigating social processes in a situation that differs from the norm will improve understanding of how social processes work in typical situations: "The exception that proves the rule."

Some qualiative researchers do question the value of gmeralizabil'ity,as most researchers understand it. The argument is that understanding the particulars of a situation in depth is an important object of inquiry in itself. In the words of sociologist Norman Denzin,

The interpretivist rejects generalization as a goal and never aims to draw randomly selected samples of human experience. . . . Every instance of social interaction . . . represents a slice from the life world that is the proper subject matter for interpretive inquiry. @enzin cited in Schofield 2002,173)

SAMPLI NG DISTRIBUTIONS

A well-designed probability sample is one that is likely to be representative of the population from which it was selected. But as you've seen, random samples are still subject to sampling error owing to chance. To deal with that problem, social researchers consider the properties of a sampling distributio4 a hypothetical distribution of a statistic across all the random samples that could be drawn from a population. Any single random sample can be thought of as one of an infinite number of random samples that, in theory could have been selected from the population. Ifwe had the finances of Gatsby and the patience of Job and were able to draw an infinite number of samples, and we calculated the same type of statistic for each of these samples, we would then have a sampling distribution. IJnderstanding sampling distributions is the foundation for understanding how statisticians can estimate sampling error.

What does a sampling distribution look like? Because a sampling distribution is based on some satistic calculated for different samples, we need to choose a statistic. Let's focus on the arithmetic average, or ntean. We will explain the calculation of the mean in Chapter 14, but you may already be familiar with it: You add up the values of all the cases and divide by the total number of cases. Let's say you draw a random sample of 500 families and find that their average (mean) family income is $58,239. Imagine that you then draw another random sample. That samplet mean family income might be $60,302. Imagine marking these two means on graph paper and then drawing more random samples and marking their means on the graph. The resulting graph would be a sampling distribution of the mean.

Exhibit 5. 10 demonstrates what happened when we did something very similar-not with an infinite number of samples and not from a large population, but through the same process using the 2012 General Social Survey (GSS) sample as if it were a population. First, we drew 50 different random samples, each consisting of 30 cases, from rJrre 2012 GSS. (The standard notation for the number of cases in each sample is z = 3 0.) Then we calculated for each random sample the approximate mean family income (approximate because the GSS does not record actual income in dollars). We then graphed the means of the 50 samples. Each bar in Exhibit 5.10 shows how many samples had a mean family income in each $5,000 category berween

140 SECTION II . FUNDAMENTALS OF RESEARCH

Mean = 63530.52 std. Dev. = 11850.96224

$40,000 $50,000 $60,000 $70,000 $80,000 $90,000 $100,000 Mean Family Income in Sample

Mean income in highest recorded category (> = $150,000) set as 246,484 based on distribution in U.S. Bureau olthe Census, Current Population Survey, Family lncomeTable,2Ol2, FINC-07. http://www.census .gov/hhevwwwcpstables/o32o13/faminci/toc.htm. Output generated from IBM SPSS Statistics, Version 21 .

Source:General Social Suruey 2012 (National Opinion Research Center [NORC] 2014).

$40,000 and $95,000. The mean for the population (the total GSS sample in this example) is $64,238, and you can see that many of the samples in the sampling distribution are close to this value, with the mean for this sampling distribution being $63,530.52-almost identical to the population mean. Although many of the sample means are close to the population mean, some are quite far from it (the lowest is actually $43,508, while the highest is $92,930). If you had calculated the mean from only one sample, it could have been anywhere in this sampling distribution, but it is unlikely to have been far from the population mean-that is, unlikely to have been close to either end (or tail) of the distribution.

Estimating Sampling Error

We dont actually observe sampling distributions in real research; researchers draw the best sample they can and then are stuckwith the results-one sample, not a distribution of samples. A sampling distribution is a theoretical distribution. However, we can use the properties of sampling distributions to calculate the amount of sampling error that was likely with the random sample used in a study. The tool for calculating sampling error is called inferential statistics

I nferential statistics:

lil'*.lli*rn'al i r: a I tfi r I s f r r

*stirnati:'tfi hr;w lil,.*l'f it is

that a statistical r*sultbax*rj *rt rLai:* trrsrn a ra***rn sfiillplfi is r*prcs**tatiu* rst

tii* p*i:irlati** fr*m viihirh th* **rtr1ic is assu r**d ks

CHAPTERs oSAMPLING 141

Exhibit 5.10 Partial Sampling Distribution: Mean Family Income (Samples of Si2e 30)

o tr o =E

,fl 6.0

Normal distrihution:

*h*t:*d li'r;* a h*11 and **nter*d ar*nnd th* pop u lati** ffl**.tl

"iuith tht

numb*r *f *aE*s +,a**ri*rs rttt in apr*rJirtable prrit*rn tn

br:th sid** *l tlt* rTls'd{1,

Bandom sampling error:

*ltt*r *nr;ts hqlt',,,,i* t n th t Brpulati*n and the raryrpie

thatr:r* rlue *iil'g tr chan** fatttr* tra*d*rrt rlrrcrr), n*i l* synt*rnatit: san:pli*# *rrrsr, fianrl*m sarnfi I i rr g err*r yy1s.y rlT r*ar1 ri*t result i* an il n r*irrs$*ntative saffi pl*,

The nra$niinrje rf samiliinU errCIr r**ultins lrrsne charc* {a*tq: rs r;a.* lt* *r;tinai*r}

statistirall'f,

Sample statistic: Thc vu lu* *t a statisi;c. sr,r*h a* ;L ffwafi , **ffiSst:t*d frr:r*

s,tmple rlata,

Population parameter:

Tha valu* rsf 'a *latisti*,

xsr,ls &* tttrt*fi"1:, **mpui*d

using t*r: d*lat*r i*rt e niirc pt:pulatir.rn;a san.til1*

statisti* is n* tstirrnl* *t a pi) tr]u I ati * n ls'dram*tr:r,

Confidence interval:

Th* rxnr;,* d*{m*d bY +.n*

trsnlid*nr:* ltr*itt: i*r a samfi!s stati*tir,

Confidence limits:

Itse t:firJ*t arzd is:v,i*r h*xnrjt ar**nrJ an **iimal* *f a p* li * lati** pararn *t* r has*d tj{1 fr **.mf}ie .ttntistit:,Thc

**nfid*rt** lituits sh*w h*'*,i r*urh **rfiirl*nt * *y'n b*

frla**oJ in th* **i.inrst*,

Mean Family lncome = $64,507

Sampling distributions for many statistics, including the mean, have a normal shape. A graph of a normal distribution looks like a bell, with one hump in the middle, centered on the population mean, and the number of cases tapering off on both sides of the mean. Note that a normal distribution is symmetric: If you folded it in half at its center (at the population mean), the two halves would match perfecdy. This shape is produced by random sampling error-variation owing purely to chance. The value of the statistic varies from sample to sample because of chance, so higher and lower values are equally likely.

The partial sampling distribution in Exhibit 5.10 does not have a completely normal shape because it involves only a small number of samples (50), each of which has only 30 cases. Exhibit 5. I 1 shows what the sampling distribution of family incomes would look like if it formed a perfecdy normal distribution-if, rather than 50 random samples, we had selected thousands of random samples.

The properties of a sampling distribution faciliate the process of statistical inference. In the sampling distribution, the most frequentvalue of the sample statistic-the statistic (such as the mean) computed from sample data-is identical to the population parameter-the statistic computed for the entire population. In other words, we can have a lot of confidence that the value at the peak ofthe bell curve represents the norm for the entire population. A population parameter also may be termed dte trae oalue for the satistic in that population. A sample statistic is an estimate of a population parameter.

In a normal distribution, a predictable proportion of cases fall within certain ranges. Inferential statistics takes advantage of this feature and allows researchers to estimate how likely it is that, given a particular sample, the true population value will be within some range of the statistic. For example, a statistician might conclude from a sample of 30 families, "We cmbe 95"/o confident that the true mean family income in the total population is between $39,037 md 589,977 ." The interval from $39,037 to $89,977 would then be called the 95o/o confidence interval for tbe mean. The lower (S39,037) and upper ($89,977) bounds of this interval are termed the confidence limits. Exhibit 5.11 marks such confidence limits, indi- cating the range that encompasses 95% of the area under the normal curve; 95% of all sample means would fall within this range, as does the mean of our hypothetical sample of 30 cases.

Although all normal distributions have these same basic features, they differ from one another in the extent to which they cluster around the mean. A sampling distribution is more

FUNDAMENTALS OF RESEARCH

25% of total area

Lower confidence limit = 939,037

Upper confidence limit = 989,977

95% confidence intervdl = 95% of the total area under the curve

142 sEcrroN n

Exhibit 5.11 Normal Sampling Distribution: Mean Family Income

compact when it is based on larger samples. Sated another way, we can be more confident in estimates based on larger random samples because we know that a larger sample creates a more compact sampling distribution. Other confidence intervals, such as the 99% confidence interval, can be reported. As a matter of convention, statisticians use only the 95"/o,99o/o, atd 99.9"/o confidence limits to estimate the ranges of values that are likely to contain the tme value. These conventional limits reflect the conservatism inherent in classical satistical inference: Dont make an inferential statement unless you are very confident (atleast 95"/" confident) that it is correct.

Let us illustrate the imporance of sample size for you. Compare the two sampling distributions of mean family income in Exhibit 5.12. Both depict the resuls for 50 samples. However, in one study, each sample comprised 100 families, and in the other study, each sample comprised only five tamilies. Clearly, the larger samples (z = 100) result in a sampling distri- bution that is much more tighdy clustered around the true mean of the population compared

o -9 CL

E (E a rh o L- o .ct E tz

14

13

12

11

10

I I 7

6 5

4 3 2 1

0 35 36 37 38 39 40 41 42 43

Mean Family Income in Thousands of Dollars

50 samples where r = 5 for each sample

o -9 CL

Eg @ q- o Lo2 .cl E-Jz

1

50 samples where fi = 100 for each sample

l- @ C\I CD tO (g O) O - N $ !O (O l- @ O) O r N 0r) $ (o l'- O O) r- CD rt 1() (o NF r ol c\l c\l ol C\I cD cD cD cD cD cD cD (f) cD st' \t' s' $ s \t \t $ t lo ro 1r) 1r) Lo ro

Mean Family lncome in Thousands of Dollars

CHAPTER 5 . SAMPLING 143

Exhibit 5.L2 The Effect of Sampling Size on Sample Distribution

to the smaller samples (" = 5). lftre 95% confidence interval for mean family income for the entire GSS sample of 1,368 cases was $29,421to $3 1,711, an interval only about $2,300 wide. But the 95% confidence interval for the mean family income in one GSS subsample of 100 cases was much wider, with limits o{ $25,733 and $35,399. And for a subsample of only five cases, the 95% confidence interval was very broad: $14,104 to $47,028. This wide interval reflects the fact that we are less confident in this sample estimate. Because the confidence we have in statistics obtained from such small samples is so low, they give us very litde useful information about the population.

In most social science disciplines, including ours, researchers typically rely on 95o/o or 997o confidence intervals. In fact, every time you read the results of an opinion poll in the newspaper or hear about one on a news broadcast, you are really being given a confidence interval. For example, when a newspaper reports rhtt 30o/o of high school seniors have used marijuana in the past six months, the reporter will also usually add the phrase "plus or minus 4 percentage points."This is a confidence interval.These conventional confidence limits reflect the conservatism inherent in classical statistical inference. fu reseatchers, we cannot make an inferential statement (e.g., the population does or believes such and such) unless we are very confident (at least 95% confident) that it is comect.

We will explain how to calculate confidence intervals in Chapter 14 and how to express the variability in a sample estimate with a statistic called the standard erron The basic statistics that we introduce in that chapter will make it easier to understand these other statistics. If you have already completed a statistics course, you might want to turn now to Chapter 14's confidence interval section for a quick review. In any case, you should now have a sense of how researchers make inferences from a random sample of a population.

CONCLUSION

Sampling is the fundamental starting point in criminological research. Probability sampling methods allow a researcher to use the laws of chance, or probability, to draw samples from populations and maintain a standard of representativeness that can be estimated with a high degree of confidence. A sample of 1,000 or 1,500 individuals can easily be used to reliably estimate the characteristics of the population of a nation comprising millions of individuals.

But researchers do not come by representative samples easily. Well-designed samples require carefirl planning, some advance knowledge about the population to be sampled, and adherence to systematic selection procedures so that the selection procedures are not biased. And even after the sample data are collected, the researcher's ability to generalize from the sample findings to the population is not completely certain. The best the researcher can do is perform additional calculations that state the degree of confidence that can be placed in the sample statistic.

The alternatives to random, or probability-based, sampling methods are almost always much less desirable, even though they typically are less expensive. Without a method of select- ing cases likely to represent the population in which the researcher is interested, research findings will have to be carefirlly qualified. IJnrepresentative samples may help researchers undersand which aspects of a social phenomenon are important, but questions about the generalizability of this understanding are left unansrrered.

Social scientists often seek to generalize their conclusions from the population that they studied to some larger target population. The validity of generalizations of this gpe is neces- sarily uncertain, because having a representative sample of a particular population does not at all ensure that what we find will hold tnre in other populations. Nonetheless, the accumula- tion of findings from studies based on local or otherwise unrepresenative populations can provide important information about broader populations.

SECTION II . FUNDAMENTALS OF RESEARCH144

}.. . .:,ne,V i,eW.,.:kdy.t e. r,m,'S w,it,'h,e.F,,l a,S h c a,:id'S.

ll.llllliAv#IfrHiliffi....$ fi[iil i.,,il.li.......1 3:3, i .'.'.'./:j,: ::: ,"'.' : 1':'1'1: t' : :, , . .:uefisus , ,', 1,y')' , , ], ,, ,, : i,i:

:::: ::..:: ,: :: ::r:::: :..: ...., ,: ..: , : , ...: ..:.

i.'l,l..l...ll90uffidefic.tlmteffielll.l......l'..t 42,,

I

..i..l1l1.i..i.Co ffi efiep.,ii.iffiits.,i.i,.....l.lrttr42, .,,,,,.

,,, Di$proportionate :suatified

,.,,r,,,',Shmpling.'',1.3,0',,,,''

1. ,,,.Elernent$'.,.,.11$' '',': ,

: ' '' ' ,.,il..i.l....mfruffie#fiti6fil..ilffits..,l...1,...,.i.I.i..['.8l......i....'.......i.

.;.'11.....1...fUfefeUffieI....,St* .{UCS..,..;i:l1.f.4li,

],llii:i::i:ii]l:i:ii tfi$tilgei]]d1fi$iteil::],iSilmfili#lg'

1.,.11.l...U.$U p#6b#Hlllffi1$.ffilin$,.,

, .,,,, ,,,, ,1rethOdS,, '124,: :::: "',

l..i::..;.;,;.ll.Nuure$$i6ild.9..i.:.li:.iii:ii.i1iii2:,i$l:.l:...ll,.:...:.l:.:.:, , , ,. ',',, Nofmal,,distribution, :,l.Ql'

#6fifi[4.u4u....fi*ffdffiidt6ir,..............:.t+4..l.ll.i.ll...,...,'.l....i.....l.....l.,i........l..

iPfdib]dHili#::i:$dffifiIi#] l utfiaus:]i]l:::]I ::i::]ft24: ,PioporT:hrr.,st1atifib,d :,,,,,, i,:,:,:

''sampling ,. .,1'30',' ', , : ,;

Quota,sampling 134 I' ii,,' ' " Randorn,,digit,dialing (RDD) ,' 12.7 ' ,R;hdo* ""mner

table li;i7 , ' ' i,Rd$d6ffi,i...isfiffiPliil$,i:.ief.for.......'...'.......' .4'21|l:.l:.l.t:..il.l'..i.':..l.l...:...:..l.

Randomselectioh,i I2+",, ,: ,, .Re

liacle.ffi t,samplinS ,,,

l..2..s..l'...t...li...ll.i....l.....l.......l.........

Repid5i6fitaffi,......3{ffile.1......,.....1|22 , ',

.sidffi$1ilffiHli{i$tfibu[iam...lll.i.,ltl-a0'.l.....iiii.iii.ii..ii.i.i......i..iiii..ii..i.lii....l

:S:imfile.:.f* bm,sampling l.:.l':....:l::.ill2.7ii:l...ii..,.ili:.i:.ti:.:..:...l ,sfi-affi.alx. ...$df 1if ..l.. ..1 ..,['.3..,7,..;.. . . ., .,''

:. ] ,.

I tratifi e d. r11ao*.*rpliuB,...,

i.,,.1 rl'9',,

SlistematiC, bias 1'25 i ,, , ': ', , ', ::::,,,:,:,:,

iiS $teffi#fi d...fiand,om samp,lifi

$.lii....,i..li.,i. i.illi.zi9:

ffirget,i..fi,6pfi1effiUfi.l:111:;1.1::2..0ll.':1 . , ,: ,: .

.1..3,,. .

:::i!:!!!::!::: :::!:.::!::::::::...::.: :::. ::.! .:::::: . ::.::

:GlliH:;:Al:Pl"t:iEl;Ri:.::lbi:.::::ir:.iil::s:AiilMi::P:il i:i[;lNi:G:::'it::.i:ii:,'::ii:i::ll:i.;i:,::lii:ii:l::,:ifl:::4:i5.

aqr Eurmtdec ot eruor uorlelndod aqt ruo 4 a1druus e ruo.l; selururlsa asolJ ,taorl a.loldxa o} ruaqt asn uBJ elr.

lnq 'a.ruJ JeqlEJ oJB slaselep qJnS ilJeJJoJ aq p,no^,( ']asetep la^efuuuulndod E sI slql ]Etlt passen5 nod;1

ireqrunu sFp ol uo 3ue11 ipulJ no{ op terl1{ ' (s errr d u J s o p < - s J r] s rt Bt s arrrld rJJ s a p < - az rlleue) ly a a p,try[

alqurru^ eqt roJ sanndrrrsep re Sur4ool .,t.q sate]S pelp61 ar{t roJ eletraPrnru a8e.rarr.u eq} }E {ool B a{EI

isn ilal a8e;a,re ]Er{} prtr }ur111 'tasutep sFIr uI seleJ JapJnru ar{l Jo aFe;erre uB {ool e,tr.{u5 "e

oqi enaserd ot tuura nod r{r5l,vr uI CCU Eulsn sr(aa-rns

, , . elers-e8.ru1 rcej;u sru ppom a,topl guopeS[qo erucs

olp, o^Er{ srar{3r8eser plnoqs 'asnqB plFlr Jo s}uaplrul ruaierdar errjgaq ;ieq] sesec uodar o]

^rBI /q parlnbar

etr leuuosJed pupetu puu sJerlJEeJ iuonun]rs snl] ol ruuneler € rarduqj q petuesa.rd saqlapp8 pclqle aql

..t4....Xu4...,.e.r.e 44ffiil ifi.,8 t5fi.i iifiqffilissil.igd.fiuuduer,, e.,,'.+ ,.,. aluiU51.. 5.gii..,i#5i$fii:.i5 S --611, bllre ,(uU 5>lUi oi nopu8llqo Iurlql5 due'pr.{ sraqc;uaser eqt turp Jnegeq

., ....,.mX.6fi.'rfnug1...#ffi5* pue..;iu5l6 rr.nr*d auyn"i : ,Jo, acueleaard eqr aulruJetep ot,rrepJo ul CCU 3u1sn

. srt+, paldrues 5C5 ,qr teid€qi,rltt,

", l;;Hil

.f.

suuld EuqduES eruanlJul sernpacord eseqt plnor ,rroH

..lsuo affils:,5sbt{i'.. [.,,.. Ciunio*.,*jfifi....{5rbasar ot luesuoo

' '' ', ' .Io ,(uu u1 Sp,npppq roy eiqlssod sl il rytqr noit

Hf,UV3;S,3U JO SlVrNf,hl,,VONnJ . ll: NOl,,r3IS gf t

iBtup uoltelndod ro ewp elduus asr.rduror tasutup slq] seog 'parcellor

{lerurn))v eJE sproJOJ IEngJo rer{t erunssv 'p

ilesulup slq] rol srsdluue Jo trun eq] sr ]eqL1 'r

q.(cuanba-l; E

tE {ool no.,(;r no.{ lpl arut-r alqurru^ eqt saop }uqr11 'q

itesulup slqt uI oJE soseJ {ustu /aoH 'E .suorlsonb aseut

'1-

reasuv'tasetep etuts ZrcZ eqt lE {ool B a{Et 'tsrrg ' I

r€,

,'|t

t',,17

r.,I

eldoad 000'001.;ed olels e ur sroprnu +o roqLUnu oq+ sl slLlI+uropr n nl

LUOJ+ e.lEr:,ElEP Oq+ O+e,]S,,,oQ,] SOU! +UOp1 SlUlalels

uortrdUosaOoruPN alqeuEn

'selels ar+uo ro1 {1rle1rour }up}ur pup sossoullr snorJeA Jo+ sa}eJ o3uoprour pup ,lseler eurrc'plpp crqderOouuep crseq sapnlcur 11 slueuLl"redap ecrlod pue'sprocar luaulredep

q+lpoq 'snsuoc oql sB qcns 'socrnos lerc![o snorJEA ujo]+ s3,]sr]e+s lercglo se;rdLrLoc losE]pp srq_L ^ES'ElEp

salEls zLaz

uor;d!rcsaOlasElEo