Suppose you were required, as many students in social research courses are, to design and conduct a research project. Our experience teaching research courses in the social sciences and human services is that some students respond to this assignment by drawing a total blank. Others, however, grasp eagerly onto a topic, such as “the cause of drug addiction,” and rush off with total confidence that they are about to solve this enduring problem. In each case, the student is having difficulty adequately formulating a research problem. In the first case, the difficulty is in locating a problem to investigate, whereas in the second, the trouble lies in formulating a problem sufficiently specific that it is amenable to scientific research.

We assure you this problem is not unique to students. Every researcher must grapple with problem formulation. Because it is the initial step and provides the basis for the complete research project, problem formulation is of crucial importance. Many potentially serious difficulties can be avoided—or at least managed—by careful problem formulation. In this chapter, we present the major issues to be considered in problem formulation, beginning with how to select a problem on which to conduct research. Then we analyze how to refine the research question so it can be answered through research. Finally, we discuss factors relating to the feasibility of research.

4.1 SELECTING A RESEARCH PROBLEM

The first hurdle confronting a researcher is to select an appropriate topic for scientific investigation. Actually, this is not as difficult as it may first appear, because the social world around us is teeming with unanswered questions. Selecting a problem calls for some creativity and imagination, but researchers can turn to a number of places for inspiration.

Personal Interest

Research topics often are selected because a researcher has an interest in some aspect of human behavior, possibly owing to some personal experience. One social scientist, for example, conducted research on battered women and women's shelters in part because of her own earlier experience of being abused by her husband; another researcher, who had grown up in the only African-American family of a small rural town, did research on prejudice, discrimination, and the experience of minorities (Higgins & Johnson, 1988). Research sometimes focuses on behavior that is unique or bizarre, and thus compelling to some. Examples of such research abound, including studies of nudist colonies, pool hustlers, juvenile gangs, striptease dancers, burglars, homeless heroin addicts, and body piercers (Bourgois, Lettiere, & Quesada, 1997; Miller & Tewksbury, 2001; Polsky, 1967; Weinberg, 1968).

Researchers who select topics from their personal interests must be careful to demonstrate the scientific worth of their projects. Recall from Chapter 1 that the goals of scientific research are to describe, explain, predict, and evaluate. The purpose of research is to advance our knowledge, not just satisfy personal curiosity. For example, in her study of female strippers, psychologist Tania Israel (2002) was interested in learning how people adapt to a job that many consider to be deviant. Such a focus placed her research firmly in an established area of study and amplified its scientific contribution. A researcher who chooses a topic based on personal interest—especially if it deals with some unusual or bizarre aspect of human behavior—should be prepared for the possibility that others will fail to see the worth of that research. Even though Israel, as noted, established a legitimate scientific rationale for her study of strippers, she was subjected to considerable criticism by those who failed to appreciate its scientific value; critics derided her research as lacking in rigor, illegitimate, and embarrassing.

Social Problems

In selecting a topic, researchers often need look no farther than the daily newspaper, which is full of the many social problems that our society faces. Such problems as crime, delinquency, poverty, pollution, overpopulation, drug abuse, alcoholism, mental illness, sexual deviance, discrimination, and political oppression have all been popular sources of topics for social research. The Society for the Study of Social Problems—a professional organization to which many social scientists and human service providers belong—publishes the journal Social Problems, whose sole purpose is to communicate the results of scientific investigations into current social problems.

Each of these general categories of social problems encompasses a range of issues for study. Many studies, for example, focus on the sources of a problem. Others are concerned with the consequences these problems have for individuals or society. Still others deal with the outcomes of social programs and other intervention efforts intended to ameliorate these problems. People in the human services, who are routinely involved with many of these problems, can find opportunities for research that are directly related to their professional activities.

Testing Theory

Some researchers select problems on the basis of their use in testing and verifying a particular theory. We noted in Chapter 2 that theoretical concerns should be at issue to some degree in all research. Nearly all research has some implications for existing theory. Certain research topics, however, are selected specifically to test some aspect of a given theory. Many theories relevant to the human services have not been thoroughly tested. In some cases, this means we do not know how valid the theories are; in others, it means we do not know how wide the range of human behavior is to which the theory can be applied.

Prior Research

One of the most fruitful sources of research problems is prior research, because the findings of all research projects have limitations. Some questions are answered, but others always remain. In addition, new questions may be raised by the findings. It is, in fact, common for investigators to conclude research reports with a discussion of the weaknesses and limitations of the research, including suggestions for future research that follow from the findings that have been presented. Focusing on these unanswered questions, or expanding on previous research, is a good way to find research problems.

Prior research also can lead to new research problems if we have reason to question the findings of the original research. As emphasized in Chapter 2, it is imperative that we not complacently accept research findings, especially when conclusions are based on a single study, because opportunities exist for error or bias to influence results. If we have reason to suspect research findings, we have a ready-made problem on which to conduct research ourselves. One of our own students, in fact, found his topic in just this way when faced with the course assignment of conducting a research project. The student had read a research article suggesting a number of differences between the social settings in which marijuana is used and those in which alcohol is used. The student disagreed, partly because of his own experiences, believing instead that the social environments in which the two substances were used were, in fact, quite similar. He designed a study that allowed him to determine whether his hypotheses—or those presumably verified by the previous investigation—would better predict what he would observe. As it turned out, many of his hypotheses were supported by his findings.

Program Evaluation and Policy Implementation

Program evaluation focuses on assessing the effectiveness or efficiency of some policy, program, or practice. As noted in Chapter 1, program and practice effectiveness evaluations have become increasingly important activities for human service professionals. Today, agencies or organizations that fund the human services typically demand that evaluation research be conducted if funding is to be granted or continued. Such research, developed for practical reasons, can take many forms. A social agency, for example, may require some needs assessment research to gather information about its clients if it is to deliver services to them efficiently. Or, a practitioner may need to know which intervention strategy—group work, psychotherapy, behavior therapy, or some other— will be most effective with a particular problem. Prison officials need to know which criminal offenders are the riskiest to parole. Home health care workers need information about how to ensure that people will take medications as prescribed. In all these cases, the practical information required by an agency or practitioner determines the focus of the research effort. Research in Practice 4.1 presents examples of research ideas being generated by reviews of how well social policies are being implemented.

[image]

RESEARCH IN PRACTICE 4.1 Policy Planning and Development: Current Policy Implementation as a Source of Research Problems

Anne holds a master's degree in public policy and is a researcher and policy analyst for the Office of the Inspector General (OIG) of the Department of Health and Human Services (HHS), a large federal agency that administers major health and social service programs across the United States. According to Anne, the IG regularly searches for any apparent problems with the implementation of HHS programs. “Research ideas come from a variety of places—legislators, service professionals, and even IG staff. When we see a major issue that is getting a good deal of public attention, we often respond by drafting a research proposal and asking for approval from the head of our office. We are nonpartisan, so our goal is to create unbiased reports on the effectiveness of our programs and offer sound recommendations for increasing efficiencies and improving services. Legislative staffers regularly meet with IG staff for further information related to the programs we investigate and use this information in making recommendations to legislators.” So this illustrates current policy implementation—or problems with it—as a rich source of research problems.

One of the major investigations Anne led in 2009 pertained to the Indian Health Service (IHS) which provides health services to 1.9 million Americans who are members of federally recognized tribes (American Indian or Alaskan Natives) through small health clinics offering routine care (Office of Inspector General, 2009). When required services are not available through these clinics, IHS and the tribes purchase services through Contract Health Services (CHS) which are delivered primarily through hospitals. When CHS are purchased through Medicare-participating hospitals, HHS sets reimbursement caps at Medicare rates. However, the IG found that, in some cases, CHS payments paid more than Medicare rates (either because they were overpayments or because services were delivered outside of Medicare-participating hospitals). This was expensive for the government as well as for the service recipients, who were required to pay some of this overage. The research was designed to show what levels of cost savings would be created by requiring all CHS units to accept Medicare caps for reimbursements and how to strictly monitor and enforce these rates.

As Anne explains, “Working for an agency like the IG gives us a very rich set of data. We can access all medical claims for Medicare, Medicaid and other programs and performance data for any number of client groups.” With access to this set of data, Anne and her coworkers examined all CHS payments and compared all medical expense claims to established Medicare rates for similar services. They found that 22 percent of hospital claims between January and March 2008 were above Medicare rates, and that preventing overpayments to Medicare-participating providers and also capping all CHS services to Medicare rates would have created a savings of $13 million during the three-month period being investigated.

So, based on their research, they recommended that the Indian Health Service and tribes take action to prevent overpayments to Medicare-participating health care providers and that all CHS payments—including those of non-Medicare-participating providers—be capped at Medicare rates.

(Based on personal interviews conducted in January 2012.)

The ability to find problems in practice settings that could be the focus of program evaluation research is limited only by the creativity and imagination of the practitioner. This was brought home to us by two of our students during a recent social research course. They were doing a field placement in a community mental health clinic while taking our research course, so they decided to search for some problem at the clinic to serve as the focus of their research paper. These students noticed that one problem the clinic faced was the failure of clients to show up for appointments. In addition to creating difficulties in achieving effective intervention, this also resulted in an inefficient use of staff resources, because counselors were left idle because of missed appointments. The students designed a very simple investigation in which some clients were given a reminder phone call a day or so before their appointment and other clients were not contacted, as had been the previous practice. The researchers' concern was to establish whether the reminder call increased the rate at which people showed up for their appointments. After implementing this procedure for a while, the students concluded that the phone call did help, and would be a useful and efficient addition to the functioning of the agency.

Human Service Practice

The linkage between human service practice and evaluation research is obvious, but service delivery can serve as the catalyst for basic research as well. During the course of working in human service programs, practitioners confront social problems and human diversity daily as they interact with clients who are struggling with life problems. Often, human service workers encounter client behavior that is unusual or even dramatic, and it stimulates questions about human behavior and society. The protective service worker who faces child abuse and neglect may derive research questions about parent—child bonding or human development. Hospice staff may generate questions about the dying process and grieving. Although answers to these questions may have practice implications, research that addresses issues such as these also has important implications for social science theory.

It is not only the behavior or characteristics of clients that generate research questions in practice settings; the mode and process of human service practice itself may even be the topic of research. For example, the mechanisms by which some human service agencies differentially allocate services according to social class or race might be a research issue. In fact, study of the human service delivery system has generated many of the major theoretical advances in our understanding of human behavior and social environments. For instance, Glaser and Strauss (1965) based their classic studies concerning death and social worth on observations of hospital patient care. More recently, controversy has surfaced in the human services over what appears to be the intergenerational transmission of family violence, suggesting that experiencing or observing violence during childhood results in subsequent violent behavior as a parent or spouse. Not only does research into this issue have practical implications for intervention programs, it also is directly relevant to developing our understanding of learning, human development, the family, and society as well (Burgess & Youngblade, 1988).

Minorities in Research: The Political Context of Problem Selection

From the preceding discussion of how to select a research problem, one could get the impression that the problem selection process is largely a matter of personal preference. Guided by personal interest or experience, the prospective researcher identifies a worthy problem and then sallies forth in the pursuit of knowledge. Like most other types of human activity, however, problem selection cannot be explained solely in such individual terms. In fact, if we asked students in research courses why they chose the term paper topics they did, we might find that, in addition to personal interest, theoretical orientation, or practice interest, their choices were governed by factors such as these: “My instructor had a data set available on this problem.” “I got financial aid to work as a research assistant.” “I knew my prof was interested in this topic, so I hoped studying it might help me get a better grade.” In other words, issues of political efficacy can influence problem selection as well.

In the world of professional research, the situation is not unlike that of the student. The stakes are much higher, however, and the consequences much greater. Although the number of possible research problems may be infinite, the resources that society can allocate to research them are not. Research is a major societal enterprise in which universities, governmental organizations, private research corporations, and independent researchers compete with each other for limited resources. At the same time, societal forces are working to make sure that the concerns of vested interest groups receive attention from the research community. Thus, problem selection is very much a political issue, and the problems that affect minorities and other groups with little clout may not receive the research attention they deserve.

Consider the example of spousal abuse. Men have been assaulting their wives since long before there was social research. Before the 1970s, however, one would have been hard pressed to find much research on the topic. Today, the social science and human service literature is replete with studies on spousal abuse, and the research endeavor in this area has expanded into a focus on intimate-partner violence. What explains the change? There is no single answer, but a major factor has been the rise of the women's movement. Before the 1960s, women as a group had considerably less political power than they have today, and the special problems of this minority often received little research attention. Woman battering is now an important issue to the women's movement, and this politically powerful group has been able to translate its concerns into public policy. Partly because of its pressure, the government has allocated money specifically for research on intimate-partner violence. In addition, with the changing roles of women in society, more women during the past three decades have chosen to pursue careers as researchers in the social sciences. One consequence of more women in research positions is that, given the role of personal interest in the selection of research topics, women are more likely to identify woman battering as a problem warranting research investigation. As the topic gained more prominence in the social science and human service fields, editors of journals became more receptive to publishing research articles on the topic. All these factors, over time, had a snowball effect. The availability of funds, the potential for publication, and the desire to contribute knowledge to an area of public concern attracted researchers seeking problems to study to this area.

So, one major factor influencing the allocation of research funds is the existence of a powerful, articulate, and effective interest group that can push for research on a particular problem (Lally, 1977; Varmus, 1999). Other factors include the following:

[image] Support for research by influentials at the national policy-making levels

[image] Definition of a condition as a social problem by national influentials

[image] Public awareness of and concern about the condition

[image] Severity, extent, and economic costs of the condition

[image] Amount of publicity about the condition

[image] Amount of support for research on the condition by the major funding agencies

On this last point, it is important to recognize that major agencies of the government, such as the Department of Health and Human Services (DHHS) and the National Science Foundation (NSF), dispense millions of dollars for research each year. Support from congressional leaders and key personnel in these major departments is essential for problem areas to be deemed worthy of financial backing for research. Typically, funding sources publish requests for proposals (RFPs), which outline the organization's funding priorities and requirements. Researchers are invited to submit proposals for competitive consideration. Proposals may be for millions of research dollars, and the competition for funding is as intense and high pressure as any major business deal.

Given that the political process plays a major role in determining which problems are sufficiently important to warrant research attention, it is not surprising that those people who lack access to social power in our society also are those whose interests are least likely to be served by the research conducted. One recent—and particularly disturbing—example of minority status influencing funding was with AIDS research. During the early years of the spread of AIDS in the United States, almost all people contracting the virus were men who had sex with other men. Today, this has changed: Currently, only 68 percent of new HIV infections involve male-to-male sexual contact or intravenous drug use, and 31 percent result from heterosexual contact (Centers for Disease Control and Prevention, 2011). In the earlier years, however, AIDS was associated in many people's minds with two groups of people—homosexuals and intravenous drug users—who were seen as marginal and highly stigmatized by many Americans, especially those in positions of power. In fact, people widely defined AIDS for a number of years as a “gay disease,” and thus as something that most Americans need not worry about. The attitude of many was summed up by the comment of a person who was later to be a staff member at the White House: “Those poor homosexuals. They have declared war on nature, and nature is exacting an awful retribution” (Shilts, 1987, p. 311). As long as AIDS was defined as a disease of deviants, funds for research on its cause and prevention were slow in coming.

Other reasons for the delayed response to the AIDS crisis existed beyond the marginality and stigma the early victims suffered (Shilts, 1987). For instance, the Reagan administration, which entered office in 1981, expressed as policy the emphasis on smaller government and austerity in social and health programs. The first AIDS victims in the United States appeared in 1980. The competition for government funds during the early 1980s was fierce, and AIDS researchers typically lost the battle, partly because those suffering from AIDS had less clout in the policy-making process than other groups competing for dwindling funds.

In short, society's response to the minority status of those afflicted with AIDS, along with other political factors, contributed to a significant delay in attacking the problem. It wasn't until AIDS began to threaten the supply of blood available for transfusions and fear arose that AIDS was entering the heterosexual population that considerable research support was forthcoming.

The selection of research problems is thus a highly political process. Although powerful interest groups will always play a role in this process, researchers need to avoid the perceptual blinders that often hinder the ability of the powerful to see which problems are sufficiently important or serious to warrant attention. Human service professionals, whether they actually conduct research or not, can play an important role in the political issues surrounding problem selection. In the course of practice, human service providers deal directly with the poor, with those labeled as deviant, with minorities, and with the powerless. This puts these providers in a position to serve as advocates for inclusion on the societal research agenda of problems relevant to their clientele.

We have discussed numerous sources of research topics for practitioner—researchers. Although we have discussed them separately, more than one of these factors often influences a given choice of a research topic. Finding a research topic, however, is only the first step in problem formulation. The next step is to shape it into a problem that empirical research can solve.

4.2 SHAPING AND REFINING THE PROBLEM

As we mentioned, a frustrating trap in which novice researchers often become ensnarled is choosing a topic that is so broad and encompassing that, by itself, it offers little guidance in terms of how to proceed. Finding the “causes of juvenile delinquency” or the “weaknesses of the modern family” sounds intriguing, but these topics provide little direction concerning where to begin to look. The next step in the research process, then, is to begin translating a general topical interest into a precise, researchable problem by narrowing the scope of the problem to manageable proportions. A single investigation is unlikely to uncover the causes of juvenile delinquency, but it might provide some insight regarding the influence of particular variables on the emergence of particular delinquencies. Refining, narrowing, and focusing a research problem do not occur all at once, but rather form a continuous process involving a number of procedures.

Conceptual Development

In Chapter 2, we discussed the role of theories and hypotheses in the research process, pointing out that concepts are one of the central components of theories. In the refining of a research problem, one of the key steps is conceptual development: identifying and properly defining the concepts on which the study will focus. In exploratory studies, of course, we are entering areas where there is little conceptual development, and a major purpose of the research itself may be to identify and define concepts. In cases where theory and research already exist, however, some conceptual development occurs as a part of formulating a research problem. One part of this process, already discussed in Chapter 2, is to clearly define the meaning of concepts. Another part of the process is to narrow the focus of the concept so it encompasses a topic that is feasible to research in a single study. For example, practitioners in a youth home who had an interest in juvenile delinquency might ask themselves, “Are we interested in all forms of delinquent behavior or only in some types?” In reality, the concept of delinquency is an extremely broad category that includes all actions by juveniles that violate criminal or juvenile codes. We have no reason to assume that a single cause can explain all types of delinquency. The focus of the research therefore might be narrowed to include only certain behaviors, such as violence or truancy. The goal of this specification process, then, is to make clear exactly what the focus of the research effort is.

Once the key concepts are clearly defined, the next consideration is their measurability. Only concepts that are in some way measurable can be used in the research process. Eventually, of course, concepts will have to be operationalized, as discussed in Chapter 2, so any that cannot be readily measurable will have to be dropped. Measuring concepts sometimes can be difficult, as we note in more detail in Chapter 5. In fact, theories at times include concepts that are difficult to operationalize. However, if the concepts in a proposed study cannot be measured, then some modification in the project—and, possibly, in the theory—is necessary. This process of refining and developing concepts as a part of the research process illustrates a point made in Chapter 2 regarding the interplay between theory and research: Theories provide concepts and hypotheses for research, whereas research modifies theories through conceptual development. The Eye on Ethics section explores some of the ethical dimensions of selecting and conceptualizing research problems.

Review of the Literature

With concepts clarified and deemed measurable, we are ready to conduct a review of previous research that relates to our research problem. This review of the literature is a necessary and important part of the research process (Locke, Silverman, & Spirduso, 2010). We do it to familiarize ourselves with the current state of knowledge regarding the research problem and to learn how others have delineated similar problems. Unless we are planning a replication, it is unlikely that we will formulate our problem precisely like any one of these previous studies; rather, we are likely to pick up ideas from several that we can integrate to improve our own. Through reviewing the relevant literature, we can further narrow the focus of the research project and ensure that we do not unnecessarily duplicate what others have already done. Researchers undoubtedly will find that pitfalls can be avoided by learning from others' experiences. For example, suppose that one or more specific approaches to a topic have proven unproductive—that is, several studies have failed to find significant results or strong relationships. In this case, unless there is good reason to believe that these earlier studies contained methodological weaknesses, the same approach is likely to lead to failure once again. Future research is likely to be more productive if it focuses on studies that have achieved some positive results.

A thorough literature review calls for familiarity with basic library utilization skills (including how to locate books, professional journals, and public documents) as well as online search skills. To help with this important aspect of doing research, we have included Appendix A in this book, which deals with the basic information retrieval skills. This appendix shows how to find the books, journals, government documents, and other sources for finding reports of research. Even those with some experience using both the library and online searches likely will find some helpful new information in this appendix.

In a literature review, we conduct a systematic search of each research report for certain kinds of information. First, the reviewer pays attention to theoretical and conceptual issues: What concepts and theories are used, how well developed are they, and have they been subjected to empirical tests before? If the theories and concepts are well developed, they can serve as an important guide in designing the planned research and explaining the relationships between variables. If they are not well developed, one will have to rely more on personal insight and creativity. In this case researchers sometimes consider doing exploratory research, which may involve loosely structured interviews and less quantitative measuring devices, as a way of advancing conceptual and theoretical development.

[image]

EYE ON ETHICS The Ethics of Selecting and Conceptualizing Research Problems

Social research has an impact on people's lives. Sometimes it brings them advantages, but other times the consequences are negative. Yet some groups have more social and economic resources, and these resources can mean that these groups are better equipped to sponsor research that may bring them benefits. Disadvantaged groups may find an inordinately small share of research resources being devoted to their concerns. In addition, more powerful groups are more likely to control the political and economic structures that enable them to influence how research problems are conceptualized. Should a problem such as poverty or drugs be conceptualized in an individualistic way (due to the weaknesses or failings of the poor or the addicted) or a systemic way (due to breakdowns in familial, political, or economic structures)? Answers to these questions will shape how research is done and how victims of poverty and drugs will be treated. As one psychologist put it: “The ethical implications come on board when we reframe the political as personal, and in so doing become complicit in agendas that maintain oppressive and neglectful social conditions” (O'Neill 2005, p. 19).

One way of addressing this problem focuses on basic issues of defining and conceptualizing problems: Ask whose agenda is served by the research we conduct and the way in which we conduct it. This may help us decide whether we are complicit in maintaining social conditions that are viewed as negative by some groups.

A second way to address this problem is for researchers to seek out research problems that are not on the agenda of more powerful groups (Bonacich, 1990). This is essentially what happened, over a period of years, with the topic of spouse abuse discussed earlier in the chapter.

A third way to approach this problem is for researchers to listen to the voices of all the stakeholders in the research, irrespective of who is funding the research. A stakeholder is any person or group who stands to gain or lose from the research. The powerful stakeholders, of course—especially those who are funding the research—will have their voices heard. It is the less powerful stakeholders that the researcher needs to seek out and listen to. So, for example, if a state department of corrections funds a study on some aspect of prison life, then the corrections administrators will certainly have an influence on what is researched and how it is done. But other stakeholders to this research include the prisoners themselves and their families as well as the correctional officers who are in daily contact with them. What kind of research might benefit or work to the disadvantage of these groups? Social science and human service researchers who strive to make research more ethical would make efforts to ensure that the voices of the inmates and correctional officers are heard in the design and conduct of the research.

The research hypotheses, including identification of the independent and dependent variables, represent a second component of a literature review. Are the hypotheses clearly stated and testable? Are they related to the variables and hypotheses being considered in the planned study? Existing research can provide some fairly specific direction in terms of already tested relationships between independent and dependent variables.

The measurement and the operational definitions used in previous research are a third focus of a literature review. As noted, successful operationalization of concepts often is difficult. Previous work in this area is invaluable in finding workable measures for concepts. Past measures may require modification to meet current needs, of course, but making these modifications probably is easier than developing completely new measures, which is a difficult and time-consuming process.

A literature review also informs us about a fourth important element of research—the most appropriate research technique for a particular research problem. Successful approaches by others should be noted and unsuccessful approaches avoided. It is of the utmost importance that the problem determines the research technique that is used, and not the other way around. Many data-gathering techniques exist, because no single method is always best. As we note in subsequent chapters, each technique has its own strengths and weaknesses, and each is suitable for answering some questions but not others.

The sampling strategy is a fifth element of a literature review. Previous research can be useful in determining the sampling strategy to use and avoiding sampling problems that others have encountered. For example, suppose the study we propose calls for the use of mailed questionnaires. An ever present problem with mailed questionnaires is making sure that a sufficient number of people complete and return them. It would be useful for us to know what other investigators have experienced with people like those we plan to survey. Not all groups respond to mailed questionnaires with the same degree of enthusiasm. If the group we are proposing to sample has exhibited notoriously low return rates during the previous studies, we have to plan accordingly: We would probably increase the number of questionnaires mailed, and we would certainly use all available means of obtaining the highest possible response rate. Or, if we anticipate very low return rates, we may want to search for another group to study—or even consider whether this particular project is feasible at all.

Statistical technique is a sixth element of a literature review. In Chapters 5, 14, and 15, we will discuss issues relating to appropriate use of statistics. In the literature review, we must be aware of whether the appropriate procedures were used, if any inappropriate procedures were used, and what constraints the concepts, variables, and hypotheses placed on the kind of statistics that would be appropriate.

Finally, a literature review notes the findings and conclusions of the studies that are examined. Which hypotheses were confirmed, and what guidelines for future research were presented? One aspect of the findings to watch for is the effect size, which refers to how big an effect an independent variable has on a dependent variable. Although we discuss this concept more fully in Chapter 15, we need to assess whether a dependent variable is affected in only a small but measurable way, or whether the impact is dramatic (Gibbs, 1991).

A thorough literature review involves evaluating and comparing many research reports, identifying where they used similar procedures and reached similar outcomes, and where there were discrepancies between studies. This can be a complicated process, especially when hundreds of studies may be involved. It sometimes is helpful to produce a summary table, such as Table 4.1, to make comparisons. Notice that the table cites each separate study in the left-hand column, along with a brief description of the intervention model used in the study. The next three columns give information about the research design, sampling procedures, and measurement devices. Then, a column contains a summary of the results of each study. The final column on the right addresses any limitations found in the research. A systematic literature review of this sort provides the most useful information from previous studies. One can see at a glance how many studies came to similar conclusions and how commonly certain measuring devices were used. The ability to compile and summarize succinctly the features of studies in this fashion is essential to formulating a research problem and refining it into a research question that can be empirically investigated.

Units of Analysis

An important element in the process of shaping and refining a research problem is the decision regarding the unit of analysis to be investigated. Units of analysis are the specific objects or elements whose characteristics we wish to describe or explain and about which we will collect data. Although there are many units of analysis, five that are commonly used in human service research are individuals, groups, organizations, programs, and social artifacts (see Table 4.2). Different units of analysis are used in studying documents; these are discussed in Chapter 8.

TABLE 4.1 A Summary Table of a Literature Review of Research on Rural Homelessness

[image]

TABLE 4.2 Possible Units of Analysis in Research on Juvenile Delinquency

Unit of Analysis	Example	Appropriate Variables	Research Problem
Individuals	Adolescents arrested for larceny	Age, sex, prior arrests	Do males receive different penalties than females for similar offenses?
Groups	Delinquent gangs	Size, norms on drug usage	Are gangs involved in drug trafficking more violent than other gangs?
Organizations	Adolescent treatment agencies	Size, auspices, funding level	Do private agencies serve fewer minority and lower-class delinquents than public agencies?
Programs	Delinquency prevention programs	Theoretical model, type of host setting	What services are most frequently included in prevention programs?
Social artifacts	Transcripts of adjudication hearings	Number of references to victim injury	To what extent does the level of violence in the offense affect the kind of penalty imposed?

Much social research focuses on the individual as the unit of analysis. The typical survey, for example, obtains information from individuals about their attitudes or behavior. Whenever we define a population of inquiry with reference to some personal status, we are operating at the individual level of analysis. For example, unwed mothers, welfare recipients, mental patients, retarded children, and similar categories all identify individuals with reference to a status they occupy.

If we identify our unit of analysis as individuals, then it is important to recognize that the entire analysis will remain at that level. For the sake of describing large numbers of individuals, it is necessary to use summarizing statistics, such as averages. For example, we might, as part of a study of unwed mothers, note that their average age when giving birth was 16.8 years. Aggregating data in this fashion in no way changes the unit of analysis. We are still collecting our data about individuals.

Social scientists sometimes focus on social groups as their unit of analysis and collect data on some group characteristic or behavior. Some groups consist of individuals who share some social relationship with the other group members. For example, in families, peer groups, occupational groups, or juvenile gangs, the members have some sense of membership or belonging to the group. If we study families in terms of whether or not they are intact, then we are investigating the characteristics of a group—the family—and not the characteristics of individuals. Other groups of interest to social scientists are merely aggregates of individuals with no necessary sense of membership, such as census tracts, cities, states, or members of a particular social class. For example, we might study the relationship between poverty and delinquency by comparing the rates of delinquency in census tracts with low income and census tracts with high income. In this case, we have collected data regarding the characteristics of census tracts rather than data regarding individuals.

Social scientists also deal with organizations as the unit of analysis. Formal organizations are deliberately constructed groups that are organized to achieve specific goals. Examples of formal organizations include corporations, schools, prisons, unions, government bureaus, and human service agencies. For example, our experience may lead us to suspect that organizations providing substance abuse services can more effectively serve their clients if they have an open and democratic communication structure rather than a closed and rigidly stratified one. If we compare the success rates of organizations with different communication structures, then our study would use organizations, not individuals or groups, as the unit of analysis. Although individuals may experience success at overcoming substance abuse, only organizations can have a success rate.

Research in the human services also can focus on programs as the basic unit of analysis. The program may provide services for individuals, and it may exist as part of an organization. However, it is still a separate unit of analysis about which data can be collected. Like organizations, programs can have success rates or be assessed in terms of overall costs. For example, one research project investigated 25 programs that provided services for pregnant and parenting teenagers (Fernandez & Ruch-Ross, 1998). This research assessed each program according to its overall success rate: A successful program was one in which the clients were more likely to stay in school or stay employed and less likely to get pregnant than were clients in the other programs. Note that a program can have a success rate (in other words, a certain proportion of their clients succeeding) but an individual can only succeed or fail (rather than show a rate of success). The researchers compared the programs to determine the characteristics of successful and unsuccessful programs. Programs might cut across a number of different organizations, such as social service agencies, in which case the unit under observation is the effectiveness of the combination of services provided by these organizations.

Finally, the unit of analysis may be social artifacts, which are simply any material products that people produce. Examples are virtually endless: newspapers, buildings, movies, books, magazines, ipads, automobiles, songs, graffiti, and so on. Of all the units of analysis, social artifacts are the least frequent focus of human service research, but as reflections of people and the society that produces the artifacts, analysis of social artifacts is useful. Books and magazines, for example, can be used as artifacts in the assessment of sex-role stereotyping. Children's books have been attacked for allegedly reinforcing traditional gender roles through their presentations of men and women (Crabb & Bielewski, 1994). Any kind of legal or administrative statute also is an artifact worthy of study. One effort, for instance, used state juvenile codes as the independent variable in a study of whether legal statutes made a difference in how courts handled juvenile cases (Grichting, 1979).

Clearly specifying the unit of analysis in research is important to avoid a serious problem: an illegitimate shift in the analysis from one unit to another. Jumping from one level to another can result in erroneous conclusions being drawn. One way this can happen is called the ecological fallacy: inferring something about individuals based on data collected at higher units of analysis, such as groups. In other words, a mismatch occurs between the unit of analysis about which data are collected and about which conclusions are drawn. Suppose, for example, a study found that census tracts with high rates of teenage drug abuse also had a large percentage of single-parent families. We might be tempted to conclude that single-parent families are a factor promoting teenage drug abuse. Such a conclusion, however, represents an illegitimate shift in the unit of analysis. The data have been collected about census tracts, which are at the group level. The conclusion being drawn, however, is at the individual level—namely, that teenage drug abusers live in single-parent families. The data, however, do not show this. In fact, the data only show the association of two rates—substance abuse and single parenthood—in census tracts. (Perhaps two-parent families are a minority in a census tract, but a high proportion of children in these families abuse drugs.) It is, of course, possible that relationships found at the group level will hold at the individual level, but they may not. In our hypothetical study, some other characteristic of census tracts may lead to both high rates of drug abuse and single-parent families. The error comes in the automatic assumption that correlations at the group level necessarily reflect relationships at the individual level.

Fallacious reasoning can occur in the opposite direction as well; in this case, it is called the reductionist fallacy: inferring something about groups or other higher levels of analysis based on data collected from individuals. Suppose we collected data from individual teenagers about their drug use and family environments and found an association—namely, that teens from single-parent families are more likely to abuse drugs. Could we then conclude that communities with high rates of single-parent families would have high rates of teenage drug abuse? The answer, once again, is that we could not draw that conclusion about the group level (communities) with any certainty, because the data we have is about social process at the individual level (what happens in the lives of individual teens). It may well be that the social process that produces high rates of drug abuse in communities is different from the social process that leads individuals to use drugs. When data are collected at one level of analysis, it is always an empirical question as to whether conclusions can be drawn from that data about other levels of analysis. A clear awareness of the unit of analysis can help ensure that we do not make such illegitimate shifts.

A final point needs to be made about the unit of analysis in contrast to the source of data. The unit of analysis refers to the element about which data are collected and inferences made, but it is not necessarily the source from which data are collected. A common example is the U.S. Census, which reports data for households. We speak of household size and income, but households don't fill out questionnaires—people do. In this case, individuals, such as the heads of households, are the source of the data, but the household is the unit of analysis about which data are collected. When the unit of analysis is something other than the individual, attention must be paid to the source of that data, because this might introduce bias into the data analysis. For example, when the household is the unit of analysis, we often collect data from one member of the household. In single-parent families, headed primarily by women, we would be gathering data mostly from women. In two-parent families, we would be obtaining data from both men and women, because either could be the head of the household. In some cases, men might be the majority of those from whom we collect data. If men tend to answer some questions differently from women, then there could be a sex bias in the results even though our unit of analysis was not linked to sex. A difference that we attribute to single-parent as compared to two-parent families may result from the fact that the former involves mostly women answering questions while the latter involves mostly men.

Reactivity

The issue of reactivity is another consideration in refining a research problem. The term reactivity refers to the fact that people can react to being studied and may behave differently than when they don't think they are being studied. In other words, the data we collect from people who know they are the objects of study might be different from the data we collect from the same people if they do not know. So, a reactive research technique changes the very thing that is being studied. Suppose, for example, that you are a parent. A researcher enters your home and sets up videotaping equipment to observe your interactions with your children. Would you behave in the same way you would if the observer were not present? You might, but most people would feel strong pressures to be “on their toes” and present themselves as good parents. You might be more forgiving of your child, for instance, or give fewer negative sanctions.

Reactivity in research can take many forms, and it is a problem for virtually all sciences. It is especially acute in social research, however, because human beings are self-conscious and aware of what is happening to them. Refining a research problem and choosing a research design are done with an eye toward reducing as much as possible the extensiveness of reactivity. We consider this in assessing the various research strategies during later chapters.

Qualitative versus Quantitative Research

Another aspect of refining a research problem is to decide whether to use one of two broad strategies toward research: qualitative research, or quantitative research. Qualitative research involves data in the form of words, pictures, descriptions, or narratives. Quantitative research uses numbers, counts, and measures of things (Berg & Lune, 2012; Wakefield, 1995). In general, two factors come into play in deciding whether to conduct qualitative or quantitative research: the state of our knowledge on a particular research topic, and the individual researcher's position regarding the nature of human social behavior. Regarding the first factor, when knowledge is sketchy or there is little theoretical understanding of a phenomenon, it may be impossible to develop precise hypotheses or operational definitions. In such cases, researchers often turn to qualitative research, because it can be more exploratory in nature. The research can be descriptive, possibly resulting in the formulation of hypotheses rather than the verification of them. When enough previous research exists on a topic, however, it may be more feasible precisely to state concepts, variables, and hypotheses. It also may be possible to develop quantifiable operational definitions of what the researcher is interested in, which then allows research to take on a more quantitative nature.

[image]

RESEARCH IN PRACTICE 4.2 Behavior and Social Environments: Feminist Research Methods: Do Males and Females Have Different “Voices”?

Positivism assumes that “one size fits all”—that there is one way to know the world, and that the diversity of people in the world is irrelevant to how we gain knowledge. One of the areas in which the debates over the scientific paradigms discussed in Chapter 2 probably have been most intense is feminist research. Feminist researchers have argued that one size does not fit all—that the diversity in the world also produces diverse ways of gaining knowledge of the world. This has led some feminist researchers to explore the nonpositivist paradigms for a more complete and complex methodology to understand human behavior (Olsen, 1994; Reinharz, 1992; Schiebinger, 1999). One profound question that feminists have raised is whether ways of knowing are gendered: Are males and females socialized to perceive the world and acquire knowledge in fundamentally different ways? Furthermore, is positivism merely one of several ways of knowing rather than the only way of knowing?

The basic argument is that, because of differences between the genders—either due to biology or socialization—there is a male model of knowledge development and a female one, and the two are, to an extent, alien to each other (DeVault, 1999; Nielson, 1989). In developing some of these ideas, Carol Gilligan (1982) used the term “voice” to mean modes of thinking about the world, and tried to describe the differences between male and female “voices.” Women emphasize the importance of relationships and the danger of being separated from others. Women see connectedness between people and feel obliged to protect and nurture those to whom they feel connected. This leads to a concern about the needs of others, but the needs of self and others exist primarily in the context of relationships with other people. So women's voices focus on the individual who is embedded in a social network. This emphasis on connectedness and relationships means that problems and people are inextricably intertwined. Neither problems nor the people they affect can be fully understood if they are separated from one another.

By contrast, men's voices speak of separation and autonomy. They emphasize independence and, to an extent, alienation—at least in the sense that people can be abstracted from their relationships, from their context, and even from their own uniqueness. For men, the separate individual has some meaning and importance and, possibly, even more value than a person who is encumbered by relationships and connections. This view treats these abstracted individuals the same, ignoring the unique needs or contexts of each person. To this extent, men's voices are abstract and formal. People and their problems can be separated.

Proponents of the male-female distinction argue that these male and female voices are fundamentally contrasting ways of perceiving and developing knowledge about the world. The distinction suggests a paradigm that differs from the positivist one, with some affinities to a variety of nonpositivist paradigms. The male voice tends to be logical and objective and to avoid or downplay feelings while focusing on accomplishing practical goals. It tends to view the world in terms of separation: The researcher extracts data from the “subject” or “respondent” and, in the interests of objectivity, avoids any personal relationship (or connectedness) between them. All contact except that necessary to collect data is avoided. The male researcher prefers a more quantitative approach, with standardized measuring instruments and procedures, on the assumption that treating all people alike will produce the most valid and objective data. A number of procedures commonly used in positivist research can be seen as efforts to strip away the context from the individual: placing them in laboratory settings; using random assignment to conditions; using aggregate responses from large-scale surveys; and isolating variables for study. All these procedures assume that the context in which a person lives is merely interfering “noise,” and that more meaningful information can be obtained without it. Through quantification, the male voice minimizes the uniqueness and subjectivity of experiences by providing summary responses of the aggregate; so the fact that Jane Doe got pregnant at age 13 under a certain set of unique and meaningful circumstances gets lost when we conclude that the average teenage pregnancy in a high school occurs at 14.7 years of age.

The female voice, on the other hand, stresses feeling, empathy, and the subjective side of life, and emphasizes connectedness: Researcher and respondent are tied together in a relationship that influences the data produced. Objectivity, in the positivist sense, is impossible, because the meaning and importance of data relate to specific individuals and their relationships with one another. The researcher and subject are partners in the relationship of producing research. Subjects are collaborators. The feminist researcher also disdains the notion of the objective, impartial researcher in favor of involvement with the topics of their research. In the feminist view of gaining truth, researcher and subject work together, although in different ways, to develop understanding; truth-seeking is a mutual, cooperative effort on their part. Research questions emphasize connectedness: keeping people in their social contexts, and studying the complexity of the whole. In short, qualitative approaches are preferred. People can be studied in their homes or where they work and play. Some research methods, such as the field methods we will discuss in Chapter 9, are better suited to emphasizing this connectedness. Feminist research also emphasizes in-depth interviews, where people have an opportunity to express the fullness and complexity of their lives in their own words.

Obviously, it is possible for men to speak with a “female” voice and for women to speak with a “male” voice. The point is that these are fundamentally different ways of knowing the world. Feminist researchers do not reject positivism; they recognize that it is different but not superior. In fact, some critics argue that researchers in the social sciences, whether male or female, generally have been trained in graduate schools with a male voice, because that is what has dominated over the years among graduate faculty. Yet, neither voice is inherently superior, and neither is complete or totally objective. Each is a valid way of gaining knowledge about the world, and each involves a perspective that has some limitations. The basic insight of feminist researchers is that all people, including researchers, are fundamentally gendered beings and—whether we admit it or not—this shapes how we perceive and think about the world, how we do research, and what research results are produced.

The second consideration in choosing between quantitative and qualitative research stems from a more fundamental controversy over the nature of human social behavior. We saw in Chapter 2 that the choice between qualitative and quantitative research is related to whether one follows positivist or nonpositivist paradigms toward science. This complicated issue, which is discussed in more detail in Chapters 2 and 9, basically involves debate over whether we can meaningfully reduce the human experience to numbers and measures. Some social scientists argue that the human experience has a subjective dimension—that is, the personal meanings and feelings that people have about themselves and what they do. These meanings or feelings cannot be captured very well through numbers or measures. Instead, narrative descriptions of people going about their daily routines or lengthy and broad-ranging interviews with them better express those meanings and feelings. Such techniques are better able to capture the critical subjective meanings that are an essential element of understanding human behavior. Quantitative research, on the other hand, provides us with much more precise statements about human behavior.

The line between qualitative and quantitative approaches is not always completely clear, and the choice between the two can be difficult. Many research projects incorporate both approaches to gain the most benefit. Research in Practice 4.2 presents some additional elaboration on this issue.

Cross.Sectional versus Longitudinal Research

In addition to deciding on the unit of analysis to investigate, refining a research problem also requires a decision about the time dimension. Here, the basic issue is whether the researcher wants a single snapshot in time of some phenomenon or an ongoing series of photographs over time. The former is called cross-sectional research , and it focuses on a cross section of a population at one point in time. Many surveys, for example, are cross-sectional in nature.

Although researchers collect all the data in cross-sectional research at one time, we can use such studies to investigate the development of some phenomena over time. For example, to study the developmental problems of children of alcoholic parents, one could select groups of children of varying ages—say, one group at age 5, another group at age 10, and a third group at age 15. By observing differences in developmental problems among these groups, we may infer that a single youngster would experience changes as he or she grew up similar to the differences observed among these three groups. Yet one of the major weaknesses of such cross-sectional studies is that we have not actually observed the changes an individual goes through; rather we have observed three different groups of individuals at one point in time. Differences among these groups may reflect something other than the developmental changes that individuals experience. Because of this disadvantage, researchers sometimes resort to the other way of handling the time issue: longitudinal studies.

Longitudinal research involves gathering data over an extended period, such as months, years, or even, in a few cases, decades. One type of longitudinal approach is the panel study , in which the same people are studied at different times. This allows us to observe the actual changes these individuals go through over time. For example, a study of the social, psychological, and familial characteristics that influence whether drug addicts can successfully remain free of drugs followed the same 354 narcotics addicts for more than 24 years, collecting data on their family experiences, employment records, and a host of other factors (Bailey et al., 1994). Another longitudinal approach is the trend study , in which different people are observed at different times. Public opinion polling and research on political attitudes often are trend studies.

Both the nature of the research problem and practical considerations typically determine the decision about whether to use a longitudinal or a cross-sectional approach. Longitudinal studies, especially panel studies, have the advantage of providing the most accurate information regarding changes over time. A research question regarding such changes, then, probably would benefit from this approach. A disadvantage of panel studies is that they can be reactive: People's responses or behavior at one time may be influenced by the fact that they have been observed earlier. For example, a person who stated opposition to abortion in one survey may be inclined to respond the same way six months later so as not to appear inconsistent or vacillating, even if his or her attitudes had changed in the interim. Another disadvantage of panel studies is that people who participated early in a panel study may not want to—or may be unable to—participate later. People die, move away, grow uninterested, or become unavailable in other ways as panel studies progress. This loss of participants can adversely affect the validity of the research findings. The disadvantages of all longitudinal studies are that they can be difficult and expensive to conduct, especially if they span a very long period of time.

Cross-sectional research is cheaper and faster to conduct, and one need not worry about the loss of participants. However, cross-sectional research may not provide the most useful data for some research questions. Thus, the decision regarding the issue of the time dimension should be based on considerations of both the nature of the research problem and practical issues. There are times, of course, when practical feasibility plays a large part in the decision.

4.3 FEASIBILITY OF A RESEARCH PROJECT

By the time researchers have selected, shaped, and refined a research problem, the problem should be sufficiently clear that a consideration of practical issues involving the feasibility of the project is in order. Practical considerations of what the research can reasonably accomplish given the time and resources available can force researchers to reduce— sometimes painfully—the scale of a project. A careful and honest appraisal of the time and money required to accomplish a project is useful in determining the feasibility of that project as envisioned, and it can reveal whether a change in goals is called for. In making a feasibility assessment, one should keep in mind a couple of axioms that apply to research projects: “Anything that can go wrong will” (Murphy's Law), and “Everything will take longer than possibly imagined.”

The practical aspects of a project's feasibility center primarily on two related concerns: time and money (Kelly & McGrath, 1988).

Time Constraints

In developing a research project, one of the major considerations is whether there will be sufficient time to adequately complete what you hope to do. In later chapters, as we consider specific research techniques, we will see how different techniques vary in terms of how much time they take. Here we want to mention some of the major factors related to time considerations.

One factor concerns the population that is the focus of the research. If that population has characteristics that are fairly widespread, then a sufficient number of people will be readily available from which to collect data. For instance, if we were studying the differing attitudes of men and women toward work-release programs for prison inmates, we could select a sample of men and women from whatever city or state we happened to be in. If, however, our study focuses on people with special characteristics that are somewhat rare, problems may arise. In general, the smaller the number of people who have the characteristics needed for inclusion in a study, the more difficult and time-consuming it will be to contact a number sufficient to make scientifically valid conclusions. For example, a study of incestuous fathers—even in a large city—may encounter problems obtaining enough cases, because relatively few such people will be openly known.

A second problem relating to time constraints involves the proper development of measuring devices. Researchers should test all techniques used for gathering data before the actual study is conducted, and this can be very time-consuming in itself. A pretest, as we saw in Chapter 1, refers to the preliminary application of the data-gathering techniques to assess their adequacy. A pilot study is a small-scale “trial run” of all the procedures planned for use in the main study. In some studies we may need to conduct several pretests as we modify data-collection devices based on the results of earlier pretests. All in all, the refining of data-gathering procedures can consume a lot of time.

A third major factor related to time considerations is the amount of time required for actual data collection, which can range from a few hours for a questionnaire administered to a group of “captive” students to the years that are necessary in many longitudinal studies. Because the amount of time required for data collection is so variable, the issue of time requires close scrutiny when addressing the question of the feasibility of a particular research design.

A fourth consideration related to the time issue is the amount of time necessary to complete the analysis of the data. In general, the less structured the data, the more time will be required for its analysis. The field notes that serve as the data for some observational studies, for example, can be very time-consuming to analyze (see Chapter 9). Likewise, videotapes collected during an experiment or during single-subject research may need several viewings before they are adequately understood (see Chapters 10 and 11). Highly structured data in quantified form, however, also can be time-consuming to analyze (see Chapters 14 and 15). Although some quantitative data can be quickly entered into a computer file if it is in a form that can be optically scanned or submitted from a website, other data must be manually entered. Even optically scanned or web data, however, require considerable advance programming and preparation before the data can actually be submitted. Also, raw data often must be manipulated and transformed before they are ready for analysis, and this can be time-consuming as well. So, the time needed for data analysis requires careful consideration owing to wide variation in the amount of time that may be necessary.

The fifth area in which time becomes a factor is the writing of the report itself. The amount of time this consumes depends on the length and complexity of the report and the skills of the investigator. Each researcher is in the best position, based on past writing experiences, to assess the amount of time he or she requires. As a final reminder, it will likely take longer than you expect.

Financial Considerations

The financial expenditures associated with a research project are another constraint on feasibility. Good research is not always expensive. In many instances, students and human service practitioners can get by with only modest costs, because data are easy to obtain, analysis procedures are simple, and the labor is either voluntary or provided at no additional charge. Even small projects, however, are likely to require money for telephone calls, typing and duplicating questionnaires, computer equipment, Internet access, and other services that can quickly stress the tight resources of a small human service agency. At the other extreme, it is not unusual for the price tag of major research and demonstration projects in the human services to run into six figures or more. For example, between 1997 and 2003, the National Institute of Justice (2003) provided annual grants of about $1 million for the development and evaluation of educational programs to assist teenagers in developing skills to avoid drug use or other forms of delinquency and to avoid becoming victims of crime. Table 4.3 shows the budget for a hypothetical survey research project of modest scale and indicates some of the general expenditure categories to consider when assessing the feasibility of any project.

The salaries of those who conduct the study are potentially the most expensive item, especially for studies that require large interviewer staffs. Such expenses include not only the interviewers' wages, but also transportation costs and living expenses, which can be sizable. Interviewers may require hours of training before they can begin to collect data. If respondents are not available, callbacks may be necessary, which further increases the cost of each interview. To get the work done in a timely fashion and ensure reliability, investigators may need to hire people or contract with an organization to code the raw data into an analyzable format.

Computer expenses also can be formidable. A few decades ago, data analysis comprised the bulk of computer costs, but now computing services of one kind or another are used for questionnaire design, project management, data collection, literature searches, and report preparation. A significant cost here is for the software packages and Internet access that actually perform these procedures, in addition to the cost of the computer itself and peripherals like printers, scanners, zip drives, and so on.

TABLE 4.3 An Illustrative Budget for Conducting Face-To-Face Interviews of 400 People

ELEMENTS OF THE INTERVIEW	COST	% OF TOTAL
Materials for Interview: print or electronic maps; interviewer manuals; printed questionnaires;	$500	2%
Interviewer Training Sessions: 2 days of training, wages for trainer and interviewers	$2,400	9.8%
Conduct Interview: Travel Costs: travel to and locate residence (inc. 60 callbacks for people not home); 1 hour per interview; $10 per completed interview	$4,000	16.3%
Conduct Interview: contact respondent; 1.5 hours per interview	$9,000	36.7%
Clerical and Data Entry Tasks: code or transcribe interviews; enter data into an electronic data file (1 hour per interview at $10 per hour)	$4,000	16.3%
Professional Supervision: coordinating and supervising all of the above tasks (160 hours at $60,000 total annual compensation)	$4,620	18.8%
TOTAL:	$24,520	100%

Another major cost consideration is expenditures for office supplies and equipment. Under this category are such items as paper, envelopes, postage, tape, printing, and the like. Paper products might seem to be inexpensive at first glance, but given the large samples that are used for some surveys, this cost can be substantial. For example, suppose we conduct a survey of approximately 500 people. Each person receives a large envelope containing a letter of introduction; a professionally printed, four-page questionnaire; and a postage-paid return envelope. In addition, all members of the sample receive a reminder letter urging them to respond, and about 250 nonrespondents receive a second questionnaire. The cost for printing, envelopes, and postage could easily exceed $2,000. Different kinds of studies present different cost issues. For example, studies based on direct observation of behavior may necessitate high-cost equipment, such as video cameras, recorders, and videotape. Unless these are already available, the project will require substantial outlays.

Providing incentives to ensure the cooperation of people in the study also may be a cost factor. This may range from giving stickers or balloons to schoolchildren for completing a questionnaire to paying respondents in recognition of the large time commitment required for a longitudinal study. A study evaluating the effectiveness of advocacy services to women leaving their abusive partners (Sullivan, 1991) illustrates participant incentive costs. Women were interviewed before the program and at 5, 10, and 20 weeks following the program. Not only was it difficult to maintain contact with this highly mobile population, completing a survey form was a low priority for these women, given the stress and disruption in their lives. To encourage the women to participate, the 46 participants were paid $10, $20, $30, and $40, respectively, for the four interviews, for a total cost of$4,600. Payment of subjects is more of an issue in experiments that require more time investment and greater commitment from participants than usually is required in most surveys or observational studies.

Dissemination of research findings also generates costs. Besides the additional printing and office supplies for preparing reports, this expense category may include travel to professional meetings to present papers. Program evaluation studies also may entail hosting workshops or conferences with sponsors and other interested parties to ensure that the findings are incorporated into the policy and intervention planning process.

Finally, some costs associated with a research project are difficult to specify. When research is conducted under the auspices of a university, a human service agency, or a research center, some organizational resources will partially or indirectly support the research. For example, money will be spent to heat and light the building where the research project is housed, but this cost is difficult to assess precisely. A major factor in awarding a research project to a particular organization may be the fact that the organization has an extensive research library, sophisticated computer facilities, or an extensive research laboratory. The organization maintains these facilities for general use, not just for a particular research project, so it is difficult to ascertain how much of the overall cost of supporting the facilities should be assigned to a given project. Consequently, the concept of “indirect cost,” typically a percentage of the total grant request, is employed to cover these real but hard-to-specify costs. Each organization negotiates its rate with the granting agency on the basis of the facilities and equipment that the organization has for research. The amount charged to indirect costs varies from 15 percent to well in excess of 100 percent of the basic grant.

Anticipating and Avoiding Problems

Problems related to time and financial considerations arise during virtually all research projects, but their impact on the outcome of the research can be minimized if they are anticipated as much as possible, especially during the planning stage, when the details of the project are easier to change. A number of steps can be taken to anticipate problems. First, learn as much as possible from the experiences of others through the studies consulted during the literature review. We mentioned earlier that finding problems other researchers encountered is one purpose of the literature review. Also, solicit personal advice from experienced researchers who might be available for consultation. A knowledgeable researcher may be able to identify potential trouble spots in the plans and suggest modifications to avoid them.

[image]

INFORMATION TECHNOLOGY IN RESEARCH Searching the Literature: Lessons from Evidence-Based Practice

Whether one is a researcher seeking to hone a research idea into a feasible project or a human service practitioner seeking the best means to help a client, efficient access to the literature is essential. In fact, literature access represents one of the linkages between practice and research, and developments along these lines in evidence-based practice serve the needs of researchers and practitioners alike.

Evidence-based practice means integrating individual human service expertise with the best available external evidence from systematic research. A major challenge for evidence-based practice is to provide a means whereby practitioners and researchers can find the evidence in a readily useable form. To meet this challenge, social scientists from around the world developed the Campbell Collaboration. It has three main review groups: social welfare, education, and criminology. Formally organized in 2000, it was inspired by a similar group that focused on research in the health field, the Cochrane Collaboration (Hannes & Claes, 2007). Although the Campbell Collaboration Library is designed for both practitioners and researchers, each may use it for different purposes. The attraction for practitioners is that the Library provides rapid access to the latest research to help answer questions such as “Howwell does the intervention I'm considering work with problems like those I'm confronting?” The researcher might want to know “How well defined is the knowledge about my topic? What measurement tools have been used? What recommendations for further research have previous studies identified?”

The Cochrane Collaboration (2003) points out that the major bibliographic databases cover less than half the world's literature for health care and are biased toward English language publications. A similar situation holds true for the human service fields. Although common online literature databases such as Sociological Abstracts or Social Work Abstracts enable the user to locate studies, the user must sort through all the studies and draw relevant conclusions. A central feature of the Campbell Collaborative is not simply that it provides access to studies, but that it includes a database of systematic reviews, which consists of full-text articles reviewing the research available on human service. Each review follows a highly structured and systematic format and covers particular content areas, such as the methodologies used on a particular area of research, the criteria for including studies in the review, and the conclusions from the research. The plan for conducting a systematic review is developed according to guidelines from the Campbell Collaboration, and the resulting review is itself peer-reviewed before it is included in the Campbell Library topics (http://www.campbellcollaboration.org/library.php). The Library can be searched by title, author, and keyword. Consequently, researchers and practitioners can have confidence in the quality and findings of each Campbell review.

Of course, having the empirical data available in readily useable form is still only one part of the equation. The other part is accessing the data, and the key to access is asking the right questions for the database search. In addition to the Campbell review, practitioners and researchers may also need to locate actual studies. Whether searching for a review or for individual research studies, evidence-based practice experts point out that a good question for a systematic search must be clear and describe exactly what is being sought—just as a good research hypothesis clearly specifies the independent and dependent variables (Sackett et al., 2000). A widely accepted evidence-based practice principle for framing search questions is the acronym “PICO,” which stands for:

[image] P—People or patient population of interest. The demographic characteristics, the problem or condition, and the particular setting (such as community or institution) should be specified.

[image] I—Intervention. The intervention should be specified as clearly as possible, such as cognitive behavioral therapy, couples counseling, or bipolar support group.

[image] C—Comparisons which will be made. This category would be included if the reviewer wishes to specifically limit inclusion of studies to those using control groups, placebos, or alternate treatments in the research design.

[image] O—Outcomes. Here the reviewer specifies the kinds of outcomes such as behavior change, prevention, reduced cost, etc., that are of interest.

For example, an evidence-based literature search related to unintended pregnancy included the following key questions (identify the PICO elements in each of these questions):

1. How effective is counseling in a clinical setting at changing knowledge, skills, and attitudes about unintended pregnancies?

2. What is the association between behaviors that support fertility desires and the prevention of unintended conceptions?

3. What are the potential harms of contraception counseling?

4. What is the cost-effectiveness of counseling in the clinical setting to prevent unintended pregnancy? (Moos, Bartholomew, and Lohr, 2003, p. 116)

The questions provide the road map for navigating through the various databases appropriate to the practitioner's interest. In addition to helping identify search terms, the questions provide guidance for including and excluding studies from the plethora of articles one is likely to encounter. (See Appendix A for databases useful to the human services that are widely available online.)

Second, obtain whatever permissions or consents may be needed early in the planning stages of the project. Depending on the people the researcher wishes to study, it may be necessary to obtain permission from them to collect data. For example, some studies are aimed at school-age children and seek to gather data while the children are in school. To protect students from undue harassment and themselves from parental complaints, school administrators frequently are cool to allowing researchers into their schools. It may take considerable time to persuade whatever authorities are involved to grant the permissions needed—if they are granted at all. It certainly is wise to obtain any needed permissions before expending effort on other phases of the project; that effort might be wasted if permissions cannot be obtained.

The final—and, perhaps, most important—suggestion for avoiding problems is to conduct a pilot study. As noted, a pilot study is a preliminary run-through, on a small scale, of all the procedures planned for the main study. For surveys, contact and interview a small part of the sample—say, 20 people. Then, analyze the data as you would for the complete project. In experiments, the researcher should run a few groups through all procedures, looking for any unexpected reactions from participants. Observational researchers should visit the observation sites and make observations as planned for the larger study. The focus is, again, on problems that might force modifications in research plans. Deal with any problems that surface during the pilot study before the main project is launched.

Given all the pitfalls a project might encounter, it is quite possible that, at some point, the researcher may conclude the project is not feasible as planned. Before calling it quits, however, give careful consideration to possible modifications that would enhance the project's feasibility. If inadequate time or money is the problem, perhaps the project can be scaled down. It might be possible to reduce the sample size or the number of hypotheses tested to make the experiment manageable. If face-to-face interviews were originally planned, consider a mailed questionnaire or even a telephone survey as cost-cutting and time-reducing measures. If the problem is with procedures, such as may occur in an experiment, consider how they might be changed so the project can proceed. The point is that a project should not be abandoned until all efforts to make it feasible have been investigated.

4.4 REVIEW AND CRITICAL THINKING

Main Points

[image] Suitable topics for research come from a variety of sources, including personal interest, social problems, theory testing, prior research, program evaluation and policy implementation, and human service practice.

[image] Political factors also influence problem selection: Powerful interest groups encourage the expenditure of research resources on issues that are of interest to them and that may not serve the interests of less advantaged or minority groups.

[image] A general topic for research must be narrowed and focused into a precise, researchable problem.

[image] An important part of refining a research problem is conceptual development: identifying and defining the concepts on which the study will focus.

[image] Reviewing previous research related to the selected topic is a crucial step in problem development and preparing to conduct a research project. This review should produce information about theoretical and conceptual issues, research hypotheses and variables, measurement and operational definitions, research techniques, sampling strategies, statistical techniques, and findings and conclusions.

[image] The units of analysis must be clearly specified as individuals, groups, organizations, programs, or social artifacts.

[image] Continual awareness of the operative unit of analysis ensures avoidance of such errors as the ecological fallacy and the reductionist fallacy.

[image] Reactivity refers to the fact that people may behave differently when they are being watched than when they are not being watched, and the effects of reactivity must be considered when shaping a research problem.

[image] Qualitative research emphasizes the description of how people experience the world; it relies on data in the form of words, pictures, descriptions, and narratives. Quantitative research uses numbers, counts, and measures to assess statistical relationships between variables.

[image] Cross-sectional research is based on data collected at one point in time, which makes accurate conclusions about trends or behavioral changes difficult. Longitudinal studies are based on data collected over a period of time and are particularly useful for studying trends or behavioral changes.

[image] Once fully refined, the practical feasibility of a proposed research project requires realistic assessment.

Important Terms for Review

cross-sectional research

ecological fallacy

longitudinal research

panel study

qualitative research

quantitative research

Critical Thinking

Critical thinking calls for the careful assessment of the nature, extent, and various facets of a problem to be addressed. So human service practitioners, policymakers, and people in their everyday lives need to devote attention to formulating, shaping, and refining a problem. It doesn't have to be a scientific problem, which is the focus of this book; it could be a problem confronting an individual, a family, or an organization. Careful and systematic thinking about a problem can pay off in a much more clear view of the problem:

1. Why is this problem considered to be a problem at all? Are there some personal interest or political issues that lead some people to see it as a problem?

2. Are the nature and terms of the problem clear and precise? Could you engage in the equivalent of what researchers call “conceptual development” to produce a clearer and sharper focus on exactly what the nature of the problem is?

3. What have others said about this problem? Is there some equivalent to what researchers call a “review of the literature” that could uncover other ideas, perspectives, or data about the problem and its solution?

4. How do people's subjective perceptions shape how they formulate a problem? Do you tend to perceive the problem differently if you take a “quantitative” approach versus a “qualitative” approach?

Exploring the Internet

Search for websites that focus on issues such as poverty or drug abuse. An initial perusal of such sites can help identify important issues for developing the research problem such as the following: “How is the concept defined?” “Does there appear to be consensus on what the concept is?” “What controversies or issues are being debated about the topic?” You also may find organizations and resources, such as measurement tools, that may prove useful during later steps in the research process. Select a topic to research, and explore some of the websites that the search locates.

The Internet can be extremely useful in discovering prior research. Many websites consist of bibliographies or include them as a special section. One example is the National Criminal Justice Reference Service. Access this site (at http://www.ncjrs.gov) and select “Victims,” which is listed under “Topics.” This will lead to a diverse listing of sources on victims of crime. Select one of the victim categories, such as “Special Populations,” and peruse some of the available sources. As another example, the Substance Abuse and Mental Health Services Administration (http://store.samhsa.gov/home) provides immediate access to a wealth of articles related to substance abuse. Try out this site by entering a combination of terms, such as “alcohol” and “violence.”

For Further Reading

Berg, B. L., & Lune, H. (2012). Qualitative Research Methods for the Social Sciences, 8th ed. Boston: Pearson. Although this book focuses primarily on how to conduct good qualitative research, it also contains a good comparison of qualitative and quantitative research and assesses when each is the most appropriate design.

Bransford, J. D. & Stein, B. S. (1995). The Ideal Problem Solver: A Guide for Improving Thinking, Learning, and Creativity, 2nd ed. New York: Freeman. Sound thinking combined with creativity clearly are important to formulating research problems. This guide assists in improving thought processes, drawing logical deductions, enhancing creativity, and even improving communication skills.

Gross, R. (1993). The Independent Scholar's Handbook. Berkeley, Calif.: Ten Speed Press. This book contains many examples of how successful scholars developed personal hunches and notions into serious research inquiries. It also is filled with practical advice about such things as obtaining resources and communicating with other researchers who share similar research interests.

Higgins, P. C. & Johnson, J. M. (1988). Personal Sociology. New York: Praeger. This book includes many illustrations of how personal life events and experiences shaped the research interests of a variety of sociologists.

Hunt, M. (1985). Profiles of Social Research: The Scientific Study of Human Interactions. New York: Basic Books. As its title implies, this book presents a series of descriptions of major research projects. Follow these projects from inception to completion, and see successful social scientists at work.

Locke, L. F., Silverman, S. J., & Spirduso, W. (2010). Reading and Understanding Research, 3rd ed. Thousand Oaks, Calif.: Sage. This is an excellent and thorough overview of how to do a good literature review and extract the appropriate information from it in an organized fashion.

Menard, S. (2002). Longitudinal Research, 2nd ed.

Newbury Park, Calif.: Sage. This book provides a readable overview of both longitudinal and cross-sectional research. It discusses when each is an appropriate design and some of the problems that arise in doing good longitudinal research.

Reinharz, S. (1992). Feminist Methods in Social Research. New York: Oxford University Press. This book is a massive compilation of examples for all types of research conducted by researchers identified as feminists. Anyone interested in conducting research from this perspective is well advised to consult this impressive work.

Class Exercises for Competency Assessment

In working through the exercises in this chapter, we suggest that you review Appendix A, which covers the basic issues involved when conducting research in a library. We also have found it helpful for students in our research classes to schedule a one- or two-hour workshop with the reference personnel to help familiarize themselves with the materials available in their library, especially the human service resource material.

4.1. [Critical Thinking] [Research & Practice] Following are several broad topic areas relevant to the human services. Select one that has the greatest personal appeal to you. Identify why this topic interests you. How do your selection and the reasons for it compare with those of other students?

a. child abuse

b. dementia

c. emotionally disturbed children

d. victims of crime

e. violence in the family

f. alcoholism and drug addiction

g. discrimination against minorities

h. homelessness

i. community living for the developmentally disabled

j. effects of unemployment

k. recipient rights for mental health clients

4.2. [Research & Practice] Use your library's resources to locate several research studies published in the professional literature on the topic you selected. Your search efforts should include—but need not be limited to—the following: a popular search engine such as Google; a specialized social science service such as socINDEX, or Social Work Abstracts; government documents through the Federal Digital System, (http://www.gpo.gov/fdsys); and the library's public catalog or a service such as WorldCAT. Be prepared to discuss with the class how you located your studies.

4.3. [Critical Thinking] [Research & Practice] For each research study you locate, do the following:

a. Indicate which factors seemed to influence the researcher's selection of topic in the study—that is, personal interest, social problems, theory testing, program evaluation and policy implementation, prior research, or practice experience.

b. What was the unit of analysis for each study? Was it the same as or different from the source of the data? If different, indicate the source. Can you think of any reasons for selecting a different unit of analysis?

c. What specific problem did the study address? From the presentation of the study, what factors influenced the final, actual problem formulation? In particular, were there issues of cost, time, or feasibility that necessitated a modification of the problem from the initial proposal?

4.4. [Ethics] [Critical Thinking] [Research & Practice] Chapter 3 presented a discussion of ethical issues in research. Did a concern for ethics influence what specific research problem emerged from an interest in studying the general topic? If so, what ethical issues confronted the researchers in studying the problem?

4.5. [Diversity] [Research & Practice] We have made the point that problem selection in research is a social process in which power, interest groups, and other factors play significant roles. One practice issue that has received considerable attention in recent years is transracial adoption—the placement of minority children, especially Native-American children, in foster and adoptive homes with white families. A quick Internet search on the key words “transracial adoption” will generate sufficient information for this exercise.

a. Have one group of students imagine that they are members of an adoptive/foster parent association that has been asked to provide suggestions to the DHHS for research priorities. What research problems would this group want to have studied?

b. Have a second group of students imagine that they are tribal leaders representing the interests of Native Americans. What priorities would they have, and what questions might this group suggest? Contrast the problems suggested by each group.