QBRDM #5

Resource1.pdf

Home >Business & Finance homework help >Management homework help >QBRDM #5

Sampling

S o far, this book has mostly addressed the planning stages of research. Now we will move into the stages of choosing what variables to observe and gathering data. In this chapter, we will investigate the issue of sampling.

This chapter looks at different sample options and examines which ones are more or less likely to offer an adequate representation of a population. We will discuss sample size, two types of sampling— probability sampling and nonprobability sampling— and several techniques for sample selection.

W H AT I S S A M P L I N G?

One of the goals of research is to draw conclusions from a sample of observed cases in a population. In research, a population is a group of people affected by a research problem or question. You, the researcher, define what the pop- ulation is that you are going to study. For example, if you are interested in examining the effects of team- building exercises with children in foster care, you may select the families in your state who are providing foster care as the population you are studying. Your sample would be drawn (selected) from this group.

In most studies it is impossible to study all pertinent cases in a population, so we must choose a sample of the elements within the population. For example, it is not possible to study every case of binge drinking on every US college campus or even on a single campus. When we select a subset from a population, this group is referred to as a sample. Sampling allows the researcher to make the best use of time, money, and other resources.

A sample is a group of elements selected from a larger population in the hope that studying this smaller group will reveal important things about the group and that it may represent the larger population of that group. Sampling, then, is the process of selecting elements from which observations will be made. People do this every day: students take their first social work class and imagine what the rest of their social work classes will be like. Or you might meet a victim of domestic

Co py ri gh t 20 19 . Ox fo rd U ni ve rs it y Pr es s.

Al l ri gh ts r es er ve d. M ay n ot b e re pr od uc ed i n an y fo rm w it ho ut p er mi ss io n fr om t he p ub li sh er , ex ce pt f ai r us es p er mi tt ed u nd er U .S . or a pp li ca bl e co py ri gh t la w.

EBSCO Publishing : eBook Collection (EBSCOhost) - printed on 10/29/2019 3:07 PM via NORTHCENTRAL UNIV AN: 1854272 ; Faulkner, Samuel S., Faulkner, Cynthia A..; Research Methods for Social Workers : A Practice-Based Approach Account: s1229530.main.eds

100 R E S E A R C H M E T H O D S F O R S O C I A L W O R K E R S

violence and make generalizations about other instances of domestic violence or domestic violence victims. While people sample daily, they rarely think about good and bad sample experiences and can therefore come to inaccurate conclusions.

Social and behavioral sciences research is often conducted on individuals or groups (e.g., hospitals, schools, and families). A sampling frame is a list of all elements or other units containing the elements in a population. For example, we are going to survey all the students living in a dorm about the quality of food on campus, but we can only obtain a list of the rooms (not the residents) in the dorm. Therefore, we will be drawing our sample from the dorm rooms, which are called enumeration units, as opposed to the individual students in each dorm room, who are the elements. An enumeration unit contains one or more units to be listed in the sampling frame.

There are times when the researcher samples different elements within a sam- pling frame. For instance, we could sample the dorm rooms about the quality of food at the university and then sample individual students on campus about the quality of the food. Both the dorm rooms and the individual students would be called sampling units. A sampling unit is a population selected for inclusion within a sampling frame. The dorm rooms are selected in the first stage of sampling and become the primary sampling units (they are also the elements in the study). The general student population becomes the secondary sampling units because they are not necessarily elements of the study as some students do not reside on campus.

R A N D O M S E L E C T I O N A N D R A N D O M A S S I G N M E N T

Researchers can derive a sample using random selection, a means of selecting a sample from a larger population in which each member of the population has an equal chance of being selected for a study, or by random assignment, the selec- tion and placement of individuals from the pool of all potential participants to either the experimental group or the control group.

For example, let’s say you want to know if studying in a group results in a better grade in research classes than does studying individually. There are four sections of a research class offered this semester. Ideally, you would want to survey all the students in these classes, but, for the purpose of this example, you randomly select half of them from the class rosters (for instance, every other person on the roster lists). This is random selection. Then, you randomly assign members into one of two groups (perhaps by drawing names out of a hat). One group (the experi- mental group) studies together, and the members of the other group (the control group) study on their own. This is random assignment.

S A M P L E S I Z E : H OW M A N Y I S E NOUG H ?

As researchers, we need to be confident that our sample is representative of the population from which it was drawn. Representativeness is assumed when

Co py ri gh t 20 19 . Ox fo rd U ni ve rs it y Pr es s.

Sampling 101

characteristics of the sample are similar to those of the population from which the sample was drawn. There are several ways that we can ensure representa- tiveness. One of the easiest ways is to have a large enough sample. We know that a very small sample can be misleading. For example, interviewing three survivors of Hurricane Katrina about their intent to return and rebuild their homes might reveal that all three do not plan to return to their hometown; however, a larger sample may reveal that while some report no intention of returning, many more plan to return and rebuild. Unfortunately, there are no hard and fast formulas to use to determine the appropriate sample size. There are, however, some guidelines that you can use. One technique that is widely utilized (and has gained widespread acceptance) is simply counting the number of variables in your study and then selecting a certain number of cases for each variable. Different researchers have different opinions on what that number of cases should be, but most people agree that between ten and twenty cases per variable is adequate. Thus, if you had ten variables, you would need a sample size of between 100 and 200 subjects. Therefore, for many researchers, the challenge is not having a large enough number of variables in the study but recruiting enough subjects to participate.

E X T E R N A L A N D I N T E R N A L VA L I D I T Y

One goal of many research studies is to apply findings beyond the group from which they were drawn. For example, you study the activities of gang members in one city to try to understand gang activity in all cities. This makes sampling an important process in conducting research. External validity (sometimes re- ferred to as generalizability) is the extent to which a study’s findings are appli- cable or relevant to a group outside the study (often the population from which the sample was drawn). The more that a study can be generalized to a larger pop- ulation, the more external validity the study has. Internal validity, in its simplest form, refers to how confident the researcher can be about the independent var- iable truly causing a change in the dependent variable (as opposed to outside influences).

Externa l Va lidity

External validity cannot be quantified in terms of a specific set of guidelines. However, it can be evaluated in light of several characteristics. One characteristic is whether the study is explained in enough detail that other researchers could duplicate the study if they wished. The more a study can be replicated, the more external validity it assumes. Two other characteristics of external validity are related to how the sampling was conducted: how the respondents of the measure were chosen and the size of the sample.

Co py ri gh t 20 19 . Ox fo rd U ni ve rs it y Pr es s.

102 R E S E A R C H M E T H O D S F O R S O C I A L W O R K E R S

Interna l Va lidity

Internal validity is different from the validity of a measure. Internal validity is a measure of the worth of the overall research design. It exists when a conclusion that A leads to or results in B is correct. When designing your research study, you need to keep in mind the following seven threats to internal validity:

1. Extraneous and widespread events that coincide in time with your study: For example, you are working with students in an after- school program to teach seventh- graders social empathy skills. How do you know that the repercussions of the aftermath of recent hurricanes, wildfires, and other natural disasters and events such as mass shootings did not affect the student’s empathy?

2. Maturation or the passage of time: The passage of time during a study (especially for studies lasting months or years) can have an effect. For example, we know that people commit less crime as they become sick or weak. Therefore, if you were researching crime among older adults, factors associated with age alone may account for a lower incidence of crime in older respondents.

3. Enhanced test- testing skills: After taking a test the first time, respondents’ performance on subsequent tests often improves. For example, imagine that we are conducting a workshop for consumers in a homeless shelter on effective job- interviewing techniques. We give participants a pretest and then provide a workshop on techniques for effective job interviewing. After the workshop, we give the participants the same test again (posttest). If they scored significantly better on the posttest, we might be tempted to argue that it was the workshop that made the difference. However, we can’t be certain that taking the pretest didn’t prepare them to do better on the posttest.

4. Instrumentation: If different measures are used for the pretest and posttest, how do we know that the posttest is not easier than the pretest? For example, an easier posttest might inflate the difference in scores and thus affect the findings of the study. In other words, the results may be inaccurately reporting that the intervention was effective. Problems involving instrumentation can also develop when a researcher uses a measure that is not measuring what it is intended to measure or that is not normed for the population to whom it was given.

5. Selection bias: Selection bias refers to the differences between groups that are being compared that occur when group members choose which group (the treatment group or nontreatment group) to be in. The differences between the members of the two groups might explain away any change that occurred after the intervention. For example, participants who self- select to be in a treatment group dealing with attitudes toward feminism may already be more politically liberal. It is

Co py ri gh t 20 19 . Ox fo rd U ni ve rs it y Pr es s.

Sampling 103

important to make sure selection bias is eliminated as much as possible through randomization.

6. Experimental mortality: This refers to subjects dropping out of a study. This is one of the most common threats to internal validity and can affect sample size (you may not have enough respondents left at the end of the study to have a meaningful finding) and generalizability (your study no longer represents the characteristics of the population your sample came from).

7. Ambiguity about the direction of causal inference: To establish causation, the independent variable must precede (cause) the change in the dependent variable. For example, studies looking at substance abuse and mental health issues have established that there is a relationship between the two variables. What is not clear is which variable precedes the other.

One of the best ways to control for threats to internal validity is to use a group to compare the study group to, which strengthens the study. By having a nonstudy group to compare against the study group, the researcher can make a stronger argument that any change in the study group is due to the intervention, not to outside influences.

P RO B A B I L I T Y S A M P L I N G

In probability sampling, each and every member of the population has a chance of being selected for the study (being included in the sample). Probability sam- pling allows the researcher to make relatively few observations and generalize from those observations to the wider population. Because everyone in the pop- ulation has an equal chance of being selected for the study, we can (theoreti- cally) assume that results from the study can be generalized to the population studied. We will examine some of the most common techniques for probability sampling.

P RO B A B I L I T Y S A M P L I N G T E C H N I QU E S

Four types of sampling techniques are used in probability sampling: simple random sampling, systematic random sampling, stratified random sampling, and cluster sampling.

In simple random sampling, each person in the population is assigned a number and then a sample is generated randomly from this population. This technique requires the compilation of a list of everyone in the population, such as all residents in a nursing home. The process for selecting the sample can be as simple as drawing numbers from a hat that then will be matched to the names

Co py ri gh t 20 19 . Ox fo rd U ni ve rs it y Pr es s.

104 R E S E A R C H M E T H O D S F O R S O C I A L W O R K E R S

on the list. When drawing a sample from a population, one can return or not return subjects to the sampling space after each draw. Returning a number to the hat after it is drawn is called sampling with replacement and is a preferred method. When one is using sampling with replacement, each selection from the population is independent of the selections already made. For example, if your population at the nursing home had twenty- five individuals (elements) to select from and you were to randomly select individual number 4, that indi- vidual would become part of your sample. Then you go back to your original twenty- five elements and draw again, leaving the number 4 as an option. If you draw number 4 again, you return it to the hat and continue to draw until you get a different number to add to your sample. Drawing a sample without returning elements to the hat is called sampling without replacement.

For systematic random sampling, every nth number is selected at random (for example, every third person or every tenth person). In essence, this is identical to simple random sampling but uses a more organized technique to produce the sample. Here, simple random sampling is always used to select the first number. For example, if you want to draw a systematic sample of 1,000 individuals from a student roster containing 100,000 names, you would divide 100,000 by 1,000 to get 100, meaning you would select every 100th name on the list. You would start with a randomly selected number between 1 and 100. For example, if you started with the number 47, you would then select the 147th name, the 247th name, the 347th name, and so on.

Stratified random sampling is a method for obtaining a greater degree of rep- resentativeness. Remember, probability sampling theory requires the researcher to select a set of elements from a population in such a way that those elements ac- curately portray the parameters of the total population from which the elements are selected. To do this, you divide your population into subgroups, or strata (for instance, by sex), then you draw the sample from each stratum using a probabi- listic procedure.

Cluster sampling (sometimes referred to as multi- stage sampling) is a method for drawing a sample from a population in two or more stages. This is used when the researcher cannot get a complete list of everyone in the population but can get complete lists within clusters of the population (such as the population of a city from a phonebook). Generally, the researcher wishes to get clusters that are as di- verse as possible, whereas in stratified random sampling the goal is to find subjects who are as similar to one another as possible. Cluster sampling is accomplished through two basic steps: listing and sampling. Listing entails constructing a list of a subset of the population. Sampling occurs within your chosen clusters. The disadvantage is that each stage of the process increases sampling error. In fact, the margin of error is larger in cluster sampling than in simple random or stratified random sampling. However, one can compensate for this error by increasing the sample size.

Perhaps you are examining child maltreatment reports by grade level in your thirteen- county region. You get a list of the schools for each county (primary

Co py ri gh t 20 19 . Ox fo rd U ni ve rs it y Pr es s.

Sampling 105

cluster list) and identify a random sample of approximately 30% from each county (secondary cluster list). See Table 7.1 for an example. You will then collect child maltreatment reports by grade level in each identified school. A sample of clusters will best represent all clusters if a large enough number is selected and if all clusters are very much alike.

Table 7.1. Example of Cluster Coding by County

County Number of Elementary Schools in Each County (Primary List)

Identified Schools (Secondary List)

Fargo 6 Jefferson Elementary Jackson Elementary

Bishop 3 Washington Elementary

Johnson 7 Lincoln Elementary Carter Elementary

Sewer 5 Clinton Elementary Bush Elementary

Lake 8 Adams Elementary Buchanan Elementary

Lincoln 10 Johnson Elementary Kennedy Elementary Lincoln Elementary

Norman 2 Roosevelt Elementary

Tulane 6 Reagan Elementary Grant Elementary

Orange 5 Taft Elementary Garfield Elementary

Camargo 5 Fillmore Elementary Harding Elementary

Fisher 9 Hoover Elementary Hayes Elementary Tyler Elementary

Newman 5 Harrison Elementary Wilson Elementary

Angel 5 Nixon Elementary Reagan Elementary

Co py ri gh t 20 19 . Ox fo rd U ni ve rs it y Pr es s.

106 R E S E A R C H M E T H O D S F O R S O C I A L W O R K E R S

S A M P L I N G E R RO R

A sampling error occurs because only part of the population is directly contacted. With any sample, differences are likely to exist between the characteristics of the sampled population and the larger group from which the sample was chosen. Sampling error can be reduced in two ways. First, the larger the sample is, the smaller the sampling error. For instance, a sample size of 10% of the population will have less sampling error than a sample size of 5% of the population because more of the original population is represented within your sample. Second, a ho- mogenous population produces samples with smaller sampling errors than does a heterogeneous population. Stratified random sampling is based on this second method. Rather than selecting your sample from the total population, you en- sure that appropriate numbers of elements are drawn from homogenous subsets of the population. This means you break the sample into smaller sections with similar qualities such as age, sex, race, and occupation. For example, you want to measure client satisfaction with the services in a large social service agency. You suspect that race is a factor in consumer satisfaction. So, you separate your participants according to race and then randomly select an appropriate number from each racial category (proportionate to the number of individuals in the cat- egory). For example, if you had one hundred whites and fifty African- Americans, you might randomly select twenty whites and ten African- Americans (one- fifth of each group) to sample. Again, stratified random sampling is used most often when a simple random sample cannot guarantee enough representation from small subgroups that are important to your study.

NO N P RO B A B I L I T Y S A M P L I N G

Social work is often conducted in settings where it is not possible to use random selection of subjects or random assignment to an experimental or comparison group. This occurs for a variety of reasons. Often, a list of possible respondents for a particular study does not exist. Also, a researcher is often only able to find subjects who are willing to volunteer for one group (such as the treatment group) as opposed to being randomly assigned. Sometimes finding participants willing to join either group can be a problem. So, a second form of sampling is available: nonprobability sampling. Any technique for selecting a sample in which every individual does not have a greater than zero chance of being selected is a nonprobability sampling technique. There are four sampling techniques used in nonprobability sampling: convenience, purposive, quota, and snowball sampling.

Convenience sampling is sampling in which one relies on available subjects. That is, you get information from any source of data you can. This is one of the most frequently used sampling techniques in social work research. Some examples of researchers who use this would be a caseworker who studies her own agency, a college professor who studies students at his college, or a researcher

Co py ri gh t 20 19 . Ox fo rd U ni ve rs it y Pr es s.

Sampling 107

who observes people in her own church. This type of sampling limits the gen- eralizability of the research to the population from which the sample is drawn. Another drawback to this method is that it can be subject to sampling error be- cause of researcher bias (selecting the sample that gives the best outcome).

Purposive sampling (also called judgmental sampling) is simply selecting a sample based on one’s knowledge of a population or drawing a sample with some predetermined characteristics in mind. For example, perhaps you are a caseworker at an agency that assists consumers in obtaining assistance with utility bills. You need to do a study for a grant that would help quantify the characteristics of your participants (such as whether they rent or own their homes, whether they are employed, their level of education, their income level). Because your study must be completed soon and it is the middle of a cold winter, you decide to select those individuals who are requesting assistance for electric bills. You feel that individuals who seek assistance for electric bills reflect the majority of your clientele, as opposed to clients who seek assistance with tele- phone payments.

Quota sampling is a means of selecting a stratified nonrandom sample in which a researcher divides a population into categories and selects a certain number (a quota) of subjects from each category. Individual subjects from each category are not selected randomly; they are usually chosen on the basis of convenience and ease. Imagine that you are studying the treatment of individuals in a mental health setting. You want to see if professionals with different professional training treat clients in different ways. So your sample includes the psychiatrists, psychologists, and social workers who are willing to participate.

In snowball sampling, the researcher starts with one or more members of the group being studied to gain access to other members of the same group through a referral system for the purpose of building the sample. Snowball sampling is most appropriate when members of a population are difficult to locate. For ex- ample, if you are studying women who have disabled children and you are having difficulty locating these mothers, you may find that once you have established a relationship with one mother, she may be willing to introduce you to other women who have disabled children. Snowball sampling is also often used when the group being studied is engaged in an activity that is illegal or considered to be deviant. For example, to study members of a gang, you need referrals to members of the gang. The strength of this type of sampling is that it creates access that allows you to increase the sample. However, because friends and other associates are usually very similar, there may be little variation in the study.

L I M I TAT I O N S O F N O N P RO B A B I L I T Y S A M P L I N G

Although nonprobability sampling may be more convenient, it is less likely to be representative of your population than probability sampling techniques. Remember that nonprobability sampling techniques do not require that every

Co py ri gh t 20 19 . Ox fo rd U ni ve rs it y Pr es s.

108 R E S E A R C H M E T H O D S F O R S O C I A L W O R K E R S

member have a greater than zero chance of being selected. Your study is affected if those not included in your sample differ in some way from the rest of the pop- ulation. Therefore, a random sample will have more generalizability than a con- venience sample or any sample where subjects are self- selecting. For example, it might be misleading to apply the findings of a researcher who only studies gangs in Southern California to other areas of the United States because of differences in population size and characteristics, gang- related laws, law enforcement capabilities, and gang prevention programs. One question to ask is “How much does the sample reflect the population from which it was drawn?” For obvious reasons, a larger sample size would have more generalizability than a smaller one because the study is taking into account more of the actual population.

So when is nonprobability sampling most useful? Some examples are pilot studies in which you are doing a trial run, agency- based research, and qualita- tive investigations in which you’re not striving for generalizability but rather to reproduce and understand real life.

C A S E S C E N A R I O

As part of your research class assignment, you and three of your fellow students have been assigned to work together in a group. You have been given the task of finding out if there is any relationship between how much alcohol students consume on weekends and their academic majors. To determine this relationship, you and your group members wait outside one of the residence halls on campus, and, as students enter and leave the building, you ask if they would be willing to complete your survey.

C R I T I C A L T H I N K I N G QU E S T IO N S

Based on the scenario just presented, answer the following questions:

1. What type of sampling design would you and your group members be utilizing?

2. Would you consider your sampling to be probability or nonprobability? State your reasons for your answer.

3. What would you consider to be an adequate sample size? How did you arrive at this number?

K E Y P O I N T S

• Sampling is the process of selecting a group of subjects from a larger population in the hope that studying this smaller group (the sample) will reveal important things about the larger group (the population) from which it was drawn.

Co py ri gh t 20 19 . Ox fo rd U ni ve rs it y Pr es s.

Sampling 109

• Probability sampling is a method of sampling in which everyone in the population has an equal chance of being randomly selected for the study and randomly assigned to either the experimental group or the comparison group.

• There are four techniques for conducting probability sampling: simple random sampling, systematic random sampling, stratified random sampling, and cluster sampling.

• Nonprobability sampling is a method for selecting a sample where every member does not necessarily have a greater than zero chance of being selected.

• There are four techniques for conducting nonprobability sampling: convenience, purposive, quota, and snowball sampling.

• Internal validity refers to how confident the researcher can be about the independent variable truly causing a change in the dependent variable. There are seven threats to internal validity: extraneous events, passage of time, testing effect, instrumentation problems, selection bias, mortality of sample, and lack of casual direction.

• External validity (referred to as generalizability) is the extent to which a study’s findings are applicable or relevant to a group outside the study. Characteristics of external validity include the ability to be duplicated by other researchers, how the respondents of the measure were chosen, and the size of the sample.

P R AC T I C E E X A M

True or Fa lse

1. In research, a population is a set of entities from which a sample can be drawn to either describe a subsection of that population or generalize information to the larger population.

2. Probability sampling means that there is a high probability someone will be selected for a study.

3. A random sample will have more generalizability than a convenience sample or any sample where subjects are self- selecting.

Mu ltiple Choice

4. Selecting subjects based on a predetermined criteria is known as a. quota sampling. b. probability sampling. c. sampling frame. d. sampling error. e. none of the above.

Co py ri gh t 20 19 . Ox fo rd U ni ve rs it y Pr es s.

110 R E S E A R C H M E T H O D S F O R S O C I A L W O R K E R S

5. External validity is also known as a. reference sampling. b. generalizability. c. sampling error. d. criterion validity. e. none of the above.

6. There are four techniques for conducting nonprobability sampling: a. convenience, purposive, quota, and snowball sampling. b. confrontive, purposeful, snowball, and external. c. random, obtrusive, intrusive, and personable. d. none of the above.

7. Snowball sampling is useful when a. a researcher doesn’t know what else to do. b. a researcher is investigating hidden or illegal activity.; c. a researcher wants to remain anonymous. d. a researcher is new in town. e. none of the above.

8. Convenience sample means that a. the researcher doesn’t want anyone to know she is involved in the re-

search process. b. the researcher doesn’t have time to collect the data herself. c. the researcher is using who she has access to. d. the researcher is afraid subjects will not be truthful with answers if

they know who is collecting data. e. none of the above.

Co py ri gh t 20 19 . Ox fo rd U ni ve rs it y Pr es s.