Week 3

Part Five

Sampling and Fieldwork

Chapter 16

Sampling Designs and Sampling Procedures

AT-A-GLANCE

I. Sampling Terminology

II. Why Sample?

A. Pragmatic reasons

B. Accurate and reliable results

C. Destruction of test units

III. Practical Sampling Concepts

A. Defining the target population

B. The sampling frame

· Sampling frames for international research

C. Sampling units

IV. Random Sampling and Nonsampling Errors

A. Random sampling error

B. Systematic sampling error

C. Less than perfectly representative samples

V. Probability versus Nonprobability Sampling

VI. Nonprobability Samples

A. Convenience sampling

B. Judgment sampling

C. Quota sampling

· Possible sources of bias

· Advantages of quota sampling

D. Snowball sampling

VII. Probability Sampling

A. Simple random sampling

B. Systematic sampling

C. Stratified sampling

D. Proportional versus disproportional strata

E. Cluster sampling

F. Multistage area sampling

VIII. What Is the Appropriate Sample Design?

A. Degree of accuracy

B. Resources

C. Time

D. Advance knowledge of the population

E. National versus local project

IX. Internet Sampling Is Unique

A. Website visitors

B. Panel samples

C. Recruited ad hoc samples

D. Opt-in lists

LEARNING OUTCOMES

1. Explain reasons for taking a sample rather than a complete census

2. Describe the process of identifying a target population and selecting a sampling frame

3. Compare random sampling and systematic (nonsampling) errors

4. Identify the types of nonprobability sampling, including their advantages and disadvantages

5. Summarize the advantages and disadvantages of the various types of probability samples

6. Discuss how to choose an appropriate sample design, as well as challenges for Internet sampling

CHAPTER VIGNETTE: Changing Pocketbook Problems for Today’s Families

Each quarter, the Gallup Corporation develops a representative sample of approximately 1,000 U.S. adults to capture public perceptions on a variety of relevant topics, including financial concerns of the family. The carefully selected sample serves as a representation of the population of adults in the U.S., and as a result, researchers can be 95 percent confident that the responses of the sample are reflective of this national population, with a sampling error of less than 3 percent. Using telephone interviews, respondents are asked to describe “the most important financial problem facing your family today,” and these open-ended responses are then coded based upon the theme of the response. Trends suggest that the most important problem can often change over time, suggesting that the financial problems facing families evolve over time.

SURVEY THIS!

Students are asked to look at the survey that asks a variety of questions of college students related to job preferences. They are asked to answer the following questions:

1. How well do the results collected from this survey represent the population of entry-level, business-oriented, recent college graduates?

2. If question one shown in the screenshot does not describe this population, describe one that you believe is better represented by this data.

3. Can the data be stratified in a way that would allow it to represent more specific populations?

4. Does this particular respondent neatly represent a common population?

RESEARCH SNAPSHOTS

· Finding Out About Work is a lot of Work!

What do people do for work? How long does it take them to get there? What do they earn? These and many other questions are critically important for United States economists and social scientists. The U.S. Census Bureau and the Bureau of Labor Statistics have jointly asked these questions, every month, for almost 70 years. The Current Population Survey (CPS) uses a panel sample of 60,000 households, surveying them for several months. The cost of conducting the CPS is millions of dollars. This accuracy has resulted in the CPS to be considered one of the standards by which other household surveys are conducted.

· How Much Does Your Prescription Cost? It Depends on Who You Buy It From

Most people would expect that their prescriptions would cost about the same, no matter where they buy them. The attorney general of the state of Michigan commissioned a targeted survey of 200 pharmacies to capture drug prescription costs around the state. Since the sample was drawn purposely, there was confidence that the survey would lead to some fruitful insights. Prices for the same prescription could vary as much as $100, even among pharmacies located near each other. This led to a consumer alert from the attorney general’s office—encouraging customer to shop carefully.

· American Kennel Club Tries to Keep Pet Owners Out of the Doghouse

The American Kennel Club (AKC) is an organization dedicated to promoting purebred dogs and their health and well-being as family companions. The organization commissioned a study to investigate dog ownership and the acceptance of dogs in their neighborhoods, and quota sampling was used. The sample was small—1,000 people—so the study design set quotas for completed interviews in age, sex, and geographic categories. One half of the respondents owned dogs. People without dogs tended to be concerned about dogs jumping and barking and owners not “picking up after their dogs.” Dog owners were somewhat more laid-back and happy compared to nonowners.

· Who’s at Home? Different Ways to Select Respondents

A carefully planned telephone survey often involves multistage sampling where researchers first select a sample of households and then select someone within each household to interview. One researcher conducted an analysis of various selection procedures and found the following:

· Full enumeration method – interviewer requests a list of all the adults living in the household, generates a random number, uses the number to select a name from the list, and asks to speak with that person.

· Kish method – interviewer requests the number of males and females by age, and then uses randomization to select either a male or female and a number (i.e., oldest male or third female). This method did not seem to discourage respondents by being too intrusive, and it was popular because it came close to being random.

· Interview the person who last had a birthday – generated better cooperation rates.

· New on Campus: Student Adjustment to College Life

A panel study was conducted where incoming students were assessed on their psychological traits and coping behaviors upon entry and were resurveyed at the end of their first year. The results indicate that students who engaged in negative coping behaviors or who had perfectionist tendencies would more likely have poor adjustment outcomes after the first year. Students who were optimistic and socially oriented were much more likely to adjust to the new college environment. The use of a panel approach was necessary because the researchers were interested in the change that occurred within a sample of students over time. These results can be used to develop newcomer programs or experiences that students can use to adjust to their new environment.

OUTLINE

I. SAMPLING TERMINOLOGY

· A population (universe) is any complete group (i.e., people, sales territories, stores, etc.) sharing some common set of characteristics.

· The term population element refers to an individual member of the population.

· A census is an investigation of all the individual elements making up the population—a total enumeration rather than a sample.

· A sample is a subset or some part of a larger population.

II. WHY SAMPLE?

· Pragmatic Reasons

· Applied research projects usually have budget and time constraints.

· Sampling cuts costs, reduces labor requirements, and gathers vital information quickly.

· Accurate and Reliable Results

· Most properly selected samples give results that are quite accurate,

· If the elements of a population are similar, only a small sample is necessary to accurately portray the characteristic of interest.

· A sample on occasion is more accurate than a census – interviewer mistakes, tabulation errors, and other nonsampling errors may increase during a census because of the increased volume of work.

· Destruction of Test Units

· Many research projects, especially those in quality-control testing, require the destruction of the items being tested.

· For example, if the manufacturer of firecrackers wished to find out whether each unit met a specific production standard, there would be no product left after testing.

III. PRACTICAL SAMPLING CONCEPTS

· Defining the Target Population

· Once the decision to sample has been made, the first question concerns identifying the target population.

· What is the relevant population?

· In many cases this is easy to answer, but in other cases, the decision may be difficult.

· For consumer-related research, the appropriate population element frequently is the household rather than an individual member of the household.

· At the outset of the sampling process it is vitally important to carefully define the target population so that the proper source from which the data are to be collected can be identified.

· To implement the sample in the field, tangible characteristics should be used to define the population.

· The Sampling Frame

· In practice, the sample will be drawn from a list of population elements called a sampling frame, which is a list of elements from which the sample may be drawn.

· The sampling frame is also called the working population, because these units will eventually provide units involved in the analysis.

· Some firms, called sampling services or list brokers, specialize in providing lists or databases that include the names, addresses, phone numbers, and e-mail addresses of specific populations.

· Lists offered by these companies are often compiled from subscriptions to profession journals, credit card applications, warranty card registrations, and a variety of other sources.

· A reverse directory provides listings by city and street address or by phone number, rather than alphabetical by last name, which is useful when a researcher wishes to survey only a certain geographical area.

· A sampling frame error occurs when certain sample elements are excluded or when the entire population is not accurately represented in the sampling frame.

· Population elements can be either under- or overrepresented in a sampling frame.

· Sampling Frames for International Research

· The availability of sampling frames around the globe varies dramatically.

· Not every country’s government conducts a census of population.

· Sampling Units

· During the actual sampling process, the elements of the population must be selected according to a certain procedure.

· The sampling unit is a single element or group of elements subject to selection in the sample.

· If the target population has first been divided into units (i.e., airline flights), additional terminology must be used.

· The term primary sampling unit (PSU) is used to designate units selected in the first stage of sampling.

· If successive stages of sampling are conducted, sampling units are called secondary sampling units, or tertiary sampling units.

· When there is no list of population elements, the sampling unit is generally something other than the population element. For example, in a random digit dialing study the sampling unit will be telephone numbers.

IV. RANDOM SAMPLING AND NONSAMPLING ERRORS

· If a difference exists between the value of a sample statistic of interest and the value of the corresponding population parameter, a statistical error has occurred.

· Two basic causes of differences between statistics and parameters:

1. random sampling errors

2. systematic (nonsampling) errors

· Random Sampling Error

· Random sampling error is the difference between the sample result and the result of a census conducted using identical procedures.

· Random sampling error occurs because of chance variation in the scientific selection of sampling units.

· Because random sampling errors follow chance variations, they tend to cancel one another out when averaged.

· This means that properly selected samples are generally good approximations of the population.

· Random sampling error is a function of sample size.

· As sample size increases, random sampling error decreases

· It is possible to estimate the random sampling error that may be expected with various sample sizes.

· Systematic Sampling Error

· Systematic (nonsampling) errors result from nonsampling factors, primarily the nature of a study’s design and the correctness of execution.

· These errors are not due to chance fluctuations.

· Sample biases account for a large portion of errors in research.

· Nonsampling errors have already been discussed in Chapter 8.

· Less than Perfectly Representative Samples

· Random sampling errors and systematic errors associated with the sampling process may combine to yield a sample that is less than perfectly representative of the population.

· Additional errors will occur if individuals refuse to be interviewed or cannot be contacted.

· Such nonresponse error may also cause the sample to be less than perfectly representative.

V. PROBABILITY VERSUS NONPROBABILITY SAMPLING

· Several alternative ways to take a sample are available.

· The main alternative sampling plans may be grouped into two categories:

1. probability techniques

2. nonprobability techniques.

· In probability sampling, every element in the population has a known, nonzero probability of selection.

· The simple random sample, in which each member of the population has an equal probability of being selected, is the best-known probability sample.

· In nonprobability sampling, the probability of any particular member of the population being chosen is unknown.

· The selection of sampling units in nonprobability sampling is quite arbitrary, as researchers rely heavily on personal judgment.

· Technically, no appropriate statistical techniques exist for measuring random sampling error from a nonprobability sample.

· Therefore, projecting the data beyond the sample is, technically speaking, statistically inappropriate.

· Nevertheless, nonprobability samples are pragmatic and are used in business research.

VI. NONPROBABILITY SAMPLING

· Convenience Sampling

· Convenience sampling refers to sampling by obtaining people or units that are conveniently available.

· Researchers generally use convenience samples to obtain a large number of completed questionnaires quickly and economically, or when obtaining a sample through other means is impractical.

· Research looking for cross-cultural differences in organizational or consumer behavior typically uses convenience samples.

· Judgment Sampling

· Judgment (purposive) sampling is a nonprobability technique in which an experienced individual selects the sample based on his or her judgment about some appropriate characteristics required of the sample members.

· The consumer price index (CPI) is based on a judgment sample of market-basket items, housing costs, and other selected goods and services expected to reflect a representative sample of items consumed by most Americans.

· Test-market cities often are selected because they are viewed as typical cities whose demographic profiles closely match the national profile.

· Often used in attempts to forecast election results.

· Political and sampling experts judge which small voting districts approximate overall state returns from previous election years.

· Then, these bellwether precincts are selected as the sampling units.

· The assumption is that the past voting nature of these districts is still representative of the state’s political behavior.

· Quota Sampling

· The purpose of quota sampling is to ensure that the various subgroups in a population are represented on pertinent sample characteristics to the exact extent that the investigators desire.

· In quota sampling, the interviewer has a quota to achieve.

· Aggregating the various interview quotas yields a sample representing the desired proportion of the subgroups.

· Possible Sources of Bias

· The logic of classifying the population by pertinent subgroups is essentially sound.

· However, because respondents are selected according to a convenience sampling procedure rather than on a probability basis (as in stratified sampling) the haphazard selection of subjects may introduce bias.

· Quota samples tend to include people who are easily found, willing to be interviewed, and middle class.

· Advantages of Quota Sampling

· The major advantages over probability sampling are

· speed of data collection

· lower costs

· convenience

· Although there are many problems with this method, careful supervision of the data collection may provide a representative sample for analyzing the various subgroups within a population.

· May be appropriate when the researcher knows that a certain demographic group is more likely to refuse to cooperate with a survey (e.g., older men).

· Snowball Sampling

· Snowball sampling refers to a variety of procedures in which initial respondents are selected by probability methods, but additional respondents are then obtained from information provided by the initial respondents.

· This technique is used to locate members of rare populations by referrals.

· Reduced costs and sample sizes are clear-cut advantages of snowball sampling.

· However, bias is likely to enter into the study because a person suggested by someone also in the sample has a higher probability of being similar to the first person.

· If there are major differences between those who are widely known by others and those who are not, there may be some serious problems with this technique.

· Since the focus group is not expected to be a generalized sample, snowball sampling may be very appropriate.

VII. PROBABILITY SAMPLING

· All probability samples are based on chance selection procedure, which eliminates the bias inherent in nonprobability sampling procedures because the probability sampling process is random.

· Randomness characterizes a procedure whose outcome cannot be predicted because it depends on chance.

· It should not be thought of as unplanned or unscientific—it is the basis of all probability sampling techniques.

· There are several probability sampling techniques discussed below.

· Simple Random Sampling

· Simple random sampling is a sampling procedure that assures that each element in the population will have an equal chance of being included in the sample.

· Drawing names from a hat is a typical example of simple random sampling; each person has an equal chance of being selected.

· This process is simple because it requires only one stage of sample selection, in contrast to other, more complex probability samples.

· When populations consist of large numbers of elements, tables of random numbers or computer-generated random numbers are utilized for sample selection.

· Selecting a random sample: to use a table of random numbers, a serial number is assigned to each element of the population. Then, assuming a population of 99,999 or less, five-digit numbers are selected from the table of random numbers merely by reading the numbers in any column or row, by moving upward, downward, left, or right. A random starting point should be selected at the outset.

· The random digit dialing technique of sample selection requires that the researcher identify the exchange or exchanges of interest (the first three numbers) and then use a table of numbers to select the next four numbers.

· Systematic Sampling

· Extremely simple.

· An initial starting point is selected by a random process; then every nth number on the list is selected.

· To illustrate this procedure, suppose one wishes to take a sample of 1,000 from a list consisting of 200,000 names. Using systematic selection every 200th name from the list will be drawn. In this example, the sampling interval is 200.

· While this procedure is not actually a random selection procedure, it does yield random results if the arrangement of the items in the list is random in character.

· The problem of periodicity occurs if a list has a systematic pattern, that is, if it is not random in character.

· Stratified Sampling

· The first step of choosing strata on the basis of existing information is the same for both stratified and quota sampling.

· However, the process of selecting sampling units within the strata differs substantially.

· In stratified sampling, a subsample is drawn using simple random sampling within each stratum. This is not true with quota sampling.

· The reason for taking a stratified sample is to have a more efficient sample than would be taken on the basis of simple random sampling.

· Random sampling error will be reduced because each group is internally homogeneous but there are comparative differences between groups.

· More technically, a smaller standard error may result from this sampling because the groups will be adequately represented when strata are combined.

· Another reason for selecting a stratified sample is to ensure that the sample will accurately reflect the population on the basis of the criterion or criteria used for stratification.

· Occasionally a simple random sample yields a disproportionate number of one group or another and the representativeness of the sample could be improved.

· A researcher selecting a stratified sample will proceed as follows:

· A variable (sometimes several variables) is identified as an efficient basis for stratification.

· The variable chosen should increase the homogeneity within each stratum and increase the heterogeneity between strata.

· The stratification variable is usually a categorical variable or one easily converted into categories, that is, subgroups.

· For each separate subgroup or strata, a list of population elements must be obtained, but if a complete listing is not available, a true stratified probability sample cannot be selected.

· A table of random numbers or some other device is then used to take a separate random sample within each stratum.

· The researcher must determine how large a sample must be drawn for each stratum.

· Proportional versus Disproportional Sampling

· If the number of sampling units drawn from each stratum is in proportion to the relative population size of the stratum, the sample is a proportional stratified sample.

· In a disproportional stratified sample the sample size for each stratum is not allocated in proportion to the population size but is dictated by analytical considerations (i.e., variability in store sales volume).

· Cluster Sampling

· The purpose of cluster sampling is to sample economically while retaining the characteristics of a probability sample.

· In a cluster sample, the primary sampling unit is no longer the individual element in the population (e.g., grocery stores) but a larger cluster of elements located in proximity to one another (e.g., cities).

· The area sample is the most popular type of cluster sample.

· Cluster sampling is classified as a probability sampling technique because of either the random selection of clusters or the random selection of elements within each cluster.

· Cluster samples frequently are used when lists of the sample population are not available.

· Ideally a cluster should be as heterogeneous as the population itself—a mirror image of the population.

· A problem may arise with cluster sampling if the characteristics and attitudes of the elements within the cluster are too similar.

· This problem may be mitigated by constructing clusters composed of diverse elements and by selecting a large number of sampled clusters.

· Multistage Area Sampling

· Multistage area sampling involves two or more steps that combine some of the probability techniques already described.

· Typically, geographic areas are randomly selected in progressively smaller (lower-population) units.

· The U.S. Bureau of the Census provides maps, population information, demographic characteristics for population statistics, and so on, by several small geographical areas that may be useful in sampling.

VIII. WHAT IS THE APPROPRIATE SAMPLE DESIGN?

· A researcher who must decide on the most appropriate sample design for a specific project will identify a number of sampling criteria and evaluate the relative importance of each criterion before selecting a sampling design.

· Exhibits 16.8 and 16.9 summarize the advantages and disadvantages of each nonprobability sampling technique and for the probability sampling techniques, respectively.

· Degree of Accuracy

· The degree of accuracy required or the researcher’s tolerance for sampling and nonsampling error may vary from project to project, especially when cost savings or other considerations may be a trade-off for a reduction in accuracy.

· Resources

· The cost associated with the different sampling techniques varies tremendously.

· If the researcher’s financial and human resources are restricted, certain options will have to be eliminated.

· Managers concerned with the cost of the research versus the value of the information often will opt for cost savings from a certain nonprobability sample design rather than make the decision to conduct no research at all.

· Time

· Researchers who need to meet a deadline or complete a project quickly will be more likely to select simple, less time-consuming sample designs.

· Advance Knowledge of the Population

· In many cases, a list of population elements will not be available to the researcher.

· A lack of adequate lists may automatically rule out systematic sampling, stratified sampling, or other sampling designs, or it may dictate that a preliminary study, such as a short telephone survey using random digit dialing, be conducted to generate information to build a sampling frame for the primary study.

· National versus Local Project

· Geographic proximity of population elements will influence sample design.

· When population elements are unequally distributed geographically, a cluster sample may become much more attractive.

IX. INTERNET SAMPLING IS UNIQUE

· Internet surveys allow researchers to reach a large sample rapidly—both an advantage and a disadvantage.

· If rapid response rates are expected, the sample for an Internet survey should be metered out across all time zones.

· In addition, for some populations, people are more likely to go online during the weekend than on a weekday.

· If the researcher can anticipate a day-of-the-week effect, the survey should be keep open long enough so all sample units have the opportunity to participate in the research project.

· The ease and low cost of an Internet survey also has contributed to a flood of online questionnaires, some more formal than others.

· Another disadvantage of Internet surveys is the lack of computer ownership and Internet access among certain segments of the population.

· A sample of Internet users is only representative of Internet users, who tend to be younger, better educated, and more affluent.

· This is not to say that all Internet samples are unrepresentative of all target populations.

· Nevertheless, when using Internet surveys, researchers should be keenly aware of potential sampling problems because some members of target populations do not have Internet access.

· Website Visitors

· Many Internet surveys are conducted with volunteer respondents who visit an organization’s website intentionally or by happenstance.

· These unrestricted samples are clearly convenience samples.

· They may not be representative because of the haphazard manner by which many respondents arrived at a particular website or because of self-selection bias.

· A better technique for sampling website visitors is to randomly select sampling units.

· Randomly selecting website visitors can cause a problem.

· It is possible to over-represent the more frequent visitors to the site, and thus represents site visits rather than visitors.

· There are several programming techniques and technologies that can help accomplish more representative sampling based on site traffic (i.e., cookies, registration data, or pre-screening).

· This type of sampling is most valuable if the target population is defined as visitors to a particular website.

· Evaluation and analysis of the visitors’ perceptions and experiences of the website would be a typical survey objective with this type of sample.

· Panel Samples

· Drawing a probability sample from an established consumer panel or other pre-recruited membership panel is a popular, scientific, and effective method for drawing Internet samples.

· Typically samples from a panel yield a high response rate because panel members have already agreed to cooperate with the research organization via e-mail and the Internet.

· Because the panel has already supplied demographic characteristics and other information from previous questionnaires, researchers have the ability to select panelists based on product ownership, lifestyle, or other characteristics.

· Consider Harris Interactive Inc. Harris Interactive is an Internet survey research organization that maintains a United States panel of more than 6.5 million individuals. Because Harris Interactive knows that all demographic groups are not fully accessible via the Internet, it uses a propensity-weighting scheme to ensure that survey results are representative. The research company does parallel studies - phone as well as Internet - to test the accuracy of its Internet data gathering capabilities.

· Recruited Ad Hoc Samples

· Another means for obtaining an Internet sample is to obtain or create a sampling frame of e-mail addresses on an ad hoc basis.

· Databases containing e-mail addresses can be compiled from many sources including customer/client lists, advertising banner recruiting survey participants, online sweepstakes, pop-up windows and registration forms that must be filled out in order to gain access to a particular website.

· Researchers may contact respondents by “snail mail” or by telephone to ask for their email addresses and obtain permission for an Internet survey.

· Using offline techniques, such as random-digit dialing and a short telephone screening interview, to recruit respondents can be a very practical way to get a representative sample for an Internet survey.

· For companies anticipating future Internet research, adding an optional e-mail registration into customer relationship databases (product registration cards, telephone interactions, on-site registration, etc.) can prove to be a valuable database for sample recruitment.

· Opt-in Lists

· Another means for obtaining an Internet sample is to obtain list of e-mail addresses from individuals who opt-in, that is given permission to receive e-mail messages related to a particular topic of interest.

· It is important not to send unauthorized e-mail to respondents.

· Spamming is not tolerated by experienced Internet users and can backfire, creating a host of problems—the most extreme being complaints to the Internet service provider (ISP), which may shut down the survey site.

QUESTIONS FOR REVIEW AND CRITICAL THINKING/ANSWERS

1. If you decide whether or not you want to see a new movie or television program on the basis of the “coming attractions” or television commercial previews, are you using a sampling technique? A scientific sampling technique?

Yes, this is a method of sampling. However, this type of sampling is generally not of a scientific nature. The portions shown are generally the most interesting rather than representative portions of those movies. Although a subset of all possible television programs or parts of a movie have been observed, they are not selected with an equal chance of becoming a sample member.

2. Name some possible sampling frames for the following:

a. Electric contractors

A possible sampling frame might be obtained from a mailing list broker or membership list of a professional organization.

b. Tennis players

Obtaining a list of tennis players may be difficult. It may be possible to obtain a list of people who subscribe to Tennis magazine. This list will tend to be a list of “better than average” or regular players rather than including many beginners or powderpuff players. Systematic sampling at tennis clubs and city courts may be another alternative if personal interviews can be utilized; however, these will also be influenced by sampling frame error.

c. Dog owners

A list of dog owners may possibly be obtained from a list of individuals who have sent in a “coupon” for a dog product, or possibly obtained from a consumer panel that prescreens subjects with respect to certain demographic questions and ownership questions. In towns where owners must purchase a license, the city government may also be a source of names.

d. Foreign-car owners

This may be obtained from automobile registrations.

e. Wig and hair goods retailers

This may be obtained from a mailing list broker.

f. Minority-owned businesses

Most states have some record of minority-owned businesses.

g. Men over six feet tall

This information may be obtained from driver’s license files in various states.

3. Describe the differences between a probability sample and a nonprobability sample.

Briefly, in every probability sample, elements in the population have a known, non-zero probability of selection. In non-probability sampling, the probability of any particular member of the population being chosen is unknown. To utilize statistical procedures, probability sampling is necessary. This is discussed in greater detail in the text.

4. In what type of situation is conducting a census more appropriate than sampling? When is sampling more appropriate than taking a census?

Of course, a researcher investigating a population with an extremely small number of population elements may elect to conduct a census rather than a sample because the cost, manpower, and time drawbacks are relatively insignificant. Thus, a company concerned with programmer satisfaction with its personal computer networking system may not have any pragmatic reason for avoiding in-house circulation of a questionnaire to all 25 of its employees. However, in most situations there are many pragmatic reasons for sampling, especially cost, resource, and time advantages. Further, sampling can be very accurate, and in some cases more accurate than a census.

5. Comment on the following sampling designs:

a. A citizen’s group interested in generating public and financial support for a new university basketball arena prints a questionnaire in area newspapers. Readers return the questionnaires by mail.

Those most interested in basketball or those most opposed to the financial program will be most likely to spot the questionnaire and return it. This method of self-selection will probably cause the sample to bias toward the two extreme groups.

b. A department store that wishes to examine whether the store is losing or gaining customers draws a sample from its list of credit card holders by selecting every tenth name.

While the idea of selecting every 10th name in a systematic fashion is fine, it must be questioned whether or not the credit card customers are representative of all of the store’s customers. If not, there will be a problem with this method of sampling.

c. A motorcycle manufacturer decided to research consumer characteristics by sending one hundred questionnaires to each of its dealers. The dealers would then use their sales records to track down buyers of this brand of motorcycle and distribute the questionnaires.

This is a poor sampling technique. The manufacturer has delegated the responsibility of sampling to dealers who may not know anything at all about research. Some dealers may have very accurate sales records and utilize some random process for generating 100 names. However, it is likely that some dealers would select the first 100 names or the 100 best customers or some other convenient method.

d. An advertising executive suggests that advertising effectiveness be tested in the real world. A one-page ad is placed in a magazine. One-half of the space is used for ad itself. On the other half, a short questionnaire requests that readers comment on the ad. An incentive will be given for the first thousand responses.

This is a poor sampling design. First, only those individuals who notice the ad can be in the sample. Thus, the sample is highly biased. Second, only those who “self-select” to respond are in the sample, which biases the results even more.

e. A research company obtains a sample for a focus group through organized groups such as church groups, clubs, and schools. The organizations are paid for securing respondents; no individual is directly compensated.

This technique is often used. It is convenient for the researchers because it often generates a group session where conversation will be lively. However, if the church groups, clubs, etc. are unrepresentative of the population, this technique should not be used. For example, a group having fewer working women than the population at large may not be representative.

f. A researcher suggests replacing a consumer diary panel with a sample of customers who regularly shop at a supermarket that uses optical scanning equipment. The burden of recording purchases by humans will be replaced by computerized longitudinal data.

There may be several flaws and biases resulting from a panel where shoppers are strictly single-store shoppers. Single-store shoppers may be less deal-prone than other shoppers, and they may be less prone to buy private label merchandise.

g. A banner ad on a business-oriented website reads, “Are you a large company Sr. Executive? Qualified execs receive $50 for under 10 minutes of time. Take the survey now!” Is this an appropriate way to select a sample of business executives?

Business executives are a hard to reach population. This site clearly will not yield a probability sample. However, it may be a cost-effective way to conduct an Internet survey with executives. Alternatives are sampling services that sell mailing lists or databases of names.

6. When would a researcher use a judgment, or purposive, sample?

Judgment, or purposive, samples are selected based on the researcher’s or some “expert’s” judgment because they know that some characteristic of the population element is required to fulfill the research’s purpose. The Dow-Jones average is a judgment sample, as is the Consumer Price Index. Test markets are typically selected based on an investigation of their demographic characteristics and executives’ judgment about how representative they are of the product’s national distribution.

7. A telephone interviewer asks, “I would like to ask you about race. Are you Native American, Hispanic, African-American, Asian, or white?” After the respondent replies, the interviewer says, “We have conducted a large number of surveys with people of your background, and we do not need to question you further. Thank you for your cooperation.” What type of sampling is likely being used?

This is a form of a quota sample. The firm does not waste any time with people that are not needed to fill the quota. Hence, they are filtered out as soon as possible.

8. If researchers know that consumers in various geographical regions respond quite differently to a product category, such as tomato sauce, is area sampling appropriate? Why or why not?

In area sampling, the sampling unit is no longer the individual element in the population but the geographical area. If one geographical area is unusual, it is possible that the entire sample results could be distorted when that particular area is selected. Ideally the geographical area selected is as heterogeneous as the population itself—a mirror image of the population.

9. What are the benefits of stratified sampling?

Stratified sampling divides the population into groups and subsamples from each group using probability sampling. This assures representativeness of all groups in a sample. Characteristics of each stratum can be estimated and comparisons made. When compared with other sampling techniques, stratified sampling reduces variability for the size sample.

10. What geographical units within a metropolitan area are useful for sampling?

There are numerous geographical units that are useful in sampling. The census tract, the census block, Zip Codes, Zip Codes + 4, and many others.

11. Researchers often are particularly interested in the subset of a market that contributes most to sales (for example, heavy beer drinkers or large-volume retailers). What type of sampling might be best to use with such a subset? Why?

First, the researcher must recognize there are some considerations concerning the target population and sampling frame. A consumer panel may have already identified heavy beer drinkers, data from a mailing list supplier may provide names and addresses of large volume retailers, or other lists may be otherwise available. If these lists are available, any sampling technique, preferably simple random sampling, may be utilized.

Quota sampling might be used if lists are not available. The first questions of a survey would ask about beer consumption or a like topic of interest. If there is a rare population, snowball sampling would be useful. For example, heavy users of rugby shirts would probably know others who follow this game.

12. Outline the step-by-step procedure you would use to select the following:

a. A random sample of 150 students at your university

Chances are this list is computerized and a computer program could be used to generate this list. If the student directory is used, a name could be numbered and a random numbers table could be used. More practical, however, would be the selection of a random starting point (e.g., page number, nth name on page, etc.) and then taking every nth name. One could assume that this systematic sample is equivalent to a random sample.

b. A quota sample of 50 light users and 50 heavy users of beer in a shopping mall intercept study

Step 1 will be to contact a sample of individuals. Generally, this will be every nth person who passes a certain point (e.g., bench area). Then, shoppers will be asked some qualifying questions on product usage—one or more will deal with beer consumption. Then, light and heavy users will be asked additional questions. The respondents are selected on a nonprobability basis—generally first 50 eligible respondents.

c. A stratified sample of 50 mechanical engineers, 40 electrical engineers, and 40 civil engineers from the subscriber list of an engineering journal.

Because the journal will have a list of engineers who are subscribers, this is the best sampling frame. Many magazines collect “demographic” information on subscribers. Using the assumption that separate lists are available, random samples within each list may be taken.

13. Selection for jury duty is supposed to be a totally random process. Comment on the following computer selection procedures, and determine if they are indeed random:

a. A program instructs the computer to scan the list of names and pick names that are next to those from the last scan.

During a short period, such as six months, jury notices may possibly be sent to several members of the same family. The reason is that on each computer scan, the programmed guidelines are based on a system for plucking names that are next to those from the last scan. This is not a random process. The first scan may have been a random number generation but the next one is not, nor are subsequent scans.

b. Three-digit numbers were randomly generated to select jurors from a list of licensed drivers. If the weight information listed on the license matches the random number, the person is selected.

A computer programmer who uses weight information as a factor in trying to randomly choose jurors from a list of licensed drivers has a problem. When a high weight is used, only men, and a few overweight women, will be selected for the jury. When a low weight is used, only women, and a handful of skinny men, will be selected.

c. The juror source list was obtained by merging a list of registered voters with a list of licensed drivers.

Using a list of registered voters and a list of licensed drivers will be inaccurate unless duplicate entries are eliminated. Someone who is only a driver and not a voter will have a lower chance of being selected than someone who is both a registered voter and a licensed driver.

14. [Ethics Question] To ensure a good session, a company gathers focus group members from a list of articulate participants instead of conducting random sampling The client did not inquire about sample selection when it accepted the proposal. Is this ethical?

This situation brings up two issues. The first is the researcher’s obligation to seek the truth from a representative sample. The focus group is never considered to be a representative group, but members should not be selected if they are known to be atypical. The second issue relates to the researcher’s obligation to inform the client about the nature of the research design and its weaknesses. Most researchers do not over-emphasize weaknesses, but when something is highly unusual, such as “professional focus group members” this should be disclosed.

RESEARCH ACTIVITIES

1. Phone directories are sometimes used for sampling frames. Go to http://www.reversephonedirectory.com and put in a phone number for someone you know in the reverse phone search (the number must be a listed number to get results). Comment on the accuracy of the information and the appropriateness of phone directories as a representative sample.

This used to be free, but now it appears that there is a charge to get the information. When students search a phone number, the results will tell them that there is information, but they will have to purchase it. Using directories in general poses sampling error because not all numbers are listed in the directory, and increasingly, many households no longer have a landline phone.

12. [Internet Question] Go to the U.S. Census Bureau’s home page at http://www.census.gov . Profiles of every state are available (you may find the “Quick Facts” or “Population Finder” helpful) from this website. Find the data for Louisiana. Suppose a representative sample of the state of Louisiana is used to represent the current U.S. population. How well does Louisiana represent the current U.S. population? How well does Louisiana represent California or Maine? Use the profiles of the states and of the country to form your opinion.

Students’ responses will vary because there is a considerable amount of information available from which they can describe their profiles. They may look at the demographic break down (e.g., age , sex. or race, etc.) of the states in this question and for the country overall.

CASE 16.1 Who’s Fishing?

Objective: To enable students to develop a sampling plan.

Summary: Although interest groups express concerns about the impact of saltwater fishers on the fish population, no one really knows how many people fish for recreation or how many fish they catch. How could a marketer get an accurate sample? Some suggestions include contacting residents of coastal counties using random-digit dialing, use state fishing license records, or have charter fishing businesses collect data.

Questions

1. Imagine that an agency or business has asked for help in gathering data about the number of sports anglers who fish off the coast of Georgia. What advice would you give about sampling? What method or combination of methods would generate the best results?

Sports anglers who fish off the coast of Georgia are not necessarily Georgia residents as some might travel to fish there. While the first two suggestions are reasonable, as stated in the case, using random-digit dialing will result in reaching a lot of people who do not fish or fish professionally, neither of which are in the sampling frame. However, the phone survey could screen to identify recreational fishers who have fished off the coast and agree to participate in a follow-up mail or e-mail survey. If the state issues licenses, at least addresses will be on there. Finally, possibly obtaining a list of fishers using the charter businesses, or even perhaps a travel agency that coordinates such trips, could provide access to the out-of-state fishers. Finally, snowball sampling could be used once some recreational fishers are sampled.

2. What other criteria besides accuracy would you expect to consider? What sampling methods could help you meet those criteria?

Besides accuracy, there are a number of criteria that are considered when sampling: resources, time, advance knowledge of the population (i.e., availability of lists of population members), and national versus local considerations (i.e., geographic proximity of population elements). If cost is an issue, snowball sampling is economical. However, bias is likely to enter into the study because a person suggested by someone else in the sample has a higher probability of being similar to the first person. If time is an issue, a convenience sample could be used by sampling individuals who are fishing during a specific time period.

CASE 16.2 Scientific Telephone Samples

Objective: The objective of this case is to illustrate how sampling frames may be purchased from organizations that specialize in providing mailing lists for research.

Summary: STS uses a random digit dialing method for its databases of all working residential telephone exchanges in the United States.

Questions

1. Evaluate the geographic options offered by STS. Do they seem to cover all the bases?

As the case describes, STS can furnish national, state, county, and smaller samples. They provide samples based on city, ADI, MSA, PSMA, CMSA, and by Zip Codes. The data in the case can be used as a springboard for lecturing on the nature of the sample design.

2. Evaluate the STS method of random-digit dialing.

The procedure is sound. The company first checks the Central Office Codes to make sure each number in the database meets the client’s specifications for residential households. For example, 7xx numbers might be assigned to a university so these should not be in the database. Next, the company decides if weighted or unweighted sampling is going to be used. Then, the remaining three digits are added. Thus, 6xxx numbers become more specific 6428, 6590, etc. Finally, certain numbers such as 1234, 9876, are deleted because they have a high likelihood of being assigned to a business.