Week 1 Assignment

To print, please use the print page range feature within the application.

1 Beginning a Research Project

In this chapter you will learn:

1 Why knowledge of research methods is valuable for public administrators. 2 How to develop a research project. 3 About models, how, when, and why to build them and how to use them. 4 Strategies for presenting models. 5 Definitions of some common research terms, including variables and hypotheses. 6 What to consider in selecting a research topic and stating a research question.

Public administrators often ask questions that begin “how many,” “how much,” “how efficient,” “how effective,” “how adequate,” and “why.” They may want to learn something about a group of people, how much a program will cost, or what it can accomplish for each dollar spent. They need to decide how serious a problem is, whether a policy or administrative action solved a problem, what distinguishes more effective programs from less effective ones, and whether clients are satisfied with program performance. They are accountable to politicians, parents, citizens, recipients of program services, and the courts for providing public services. Public and nonprofit organization employees may also be accountable to funding agencies.

Administrators rely on data to make better decisions, to monitor results, and to examine effects. Data is just another word for information. Understanding research methods is key to gathering, using, and evaluating information appropriately. As a current or future public administrator, you know that adequate information is essential to making effective decisions.

In the role of administrator you may need to collect and summarize data and act on your findings or supervise others who do so. You may conduct studies or contract with others to perform studies for you to answer questions about programs under your jurisdiction. You may receive regular reports to monitor the performance of your organization and employees. You may read research and get ideas you wish to implement. Even if you never initiate a study, your knowledge of the research process should leave you better able to determine the adequacy of data, interpret reports, question results, and judge the value of published research.1

This text will provide you with the skills to produce information using a variety of research tools. More importantly, we hope it will provide you with the tools to make empirically informed judgments as you sort and use information produced by others to make decisions in your role as a public administrator.

Page 1 of 29Research Methods for Public Administrators

3/3/2021file:///C:/Users/Lamar%20Holmes/Desktop/VitalSource%20Bookshelf_%20Research%20Methods...

Starting a Research Project Research should begin with careful planning. Even though determining the purposes of a study and whether or not it will produce the desired information is tedious, experience has convinced us that time used to clearly define the purpose of a study and carefully critique the research plan is time well spent. Administrators who make this preliminary investment will have a better understanding of the problem being studied and will avoid conducting poorly conceived research studies and collecting useless data. In this chapter we present guidelines for better research planning. No matter who actually conducts a study, administrators who spend time defining the problem, planning the study, debating it with others, and reviewing related research markedly improve their work and experience fewer disappointments and wasted efforts.

An excellent starting point for any administrator involved with research is to understand if a decision needs to be made, when it needs to be made, the nature of that decision, and what information would be helpful to the decision makers. Focusing on the decision can help identify the true purpose of a study. In terms of timing, if it is not possible to influence the decision, then one must consider whether the study will really be of value. When considering the nature of the decision, what level of importance does the decision warrant? Trying to measure the impact of a multi-million-dollar program affecting thousands of citizens may be more important than evaluating an internal office recycling program. Are lives, jobs, or public safety at stake? Last, identifying the information decision makers need in order to make a decision is fundamental and will provide a starting point for developing research questions.

Establishing a study’s purpose goes beyond stating exactly why it is being done. An investigator must know who wants the study done, how and when the person plans to use its findings, what resources exist to support the study, and what research has been previously done on the issue. After an investigator answers these questions, she can list the research questions and decide what evidence will provide adequate answers. She also avoids planning a study that exceeds the available resources or yields information only after it is needed. It is seldom wise to plan a large study when a smaller one will obtain the necessary information. A study’s purpose evolves to become more focused and better understood as investigators and decision makers begin their work together.

Developing a Research Question

Once the administrator understands what a study can provide—and what it cannot—the researcher should begin by stating the research question. The research question, when answered, should provide information necessary to accomplish the purpose of the research. Empirical, that is, observable information is required to answer it. By definition, research involves the study of observable information, so without it no research can take place. This text stresses numerical or quantitative information; however, qualitative, that is, non-numerical information also is empirical and important and may appropriately be used to answer a research question.

Consider the research question: “To what extent does employee telecommuting improve organizational productivity, if at all?” This question does have more than one possible answer, and obtaining an answer requires empirical information. Still, this simple question can mask the amount of work that lies ahead. What is meant by “telecommuting”? How would you define and measure “productivity”? Which employees? Are all of them employees or just people holding certain positions? Does it involve the productivity of the entire organization or just certain parts of it? How

Page 2 of 29Research Methods for Public Administrators

3/3/2021file:///C:/Users/Lamar%20Holmes/Desktop/VitalSource%20Bookshelf_%20Research%20Methods...

much improvement is expected? To help limit the study’s focus, the administrator indicates the purpose of a study. A study to determine whether to adopt a change in employment procedures will be different from a study to evaluate an existing procedure or policy.

Using Models to Organize the Research Study After stating the research question and the study’s purpose, the researcher should build a preliminary research model. Investigators use models to simplify reality by identifying important items and eliminating irrelevant details.

Models consist of variables and relationships. A single variable does not constitute a model; it must be linked with another to be part of a model. Variables that are not related to at least one other variable should be eliminated from a model. Variables that are only weakly linked to others also may be eliminated. Explicit models are the words, schematics, or equations that represent the variables and their relationships. The strength of explicit models is that they give others access to the researcher’s model and allow them to critique, replicate, or improve it.

Thinking About Models

One can think about a model visually by sketching out the variables and the expected relationships among them. For example, Example 1.1 presents a model developed as part of a study to identify how to slow down the rate of increase in Medicare costs. The research question this model might be intended to answer is: “What factors are related to the increase in Medicare costs?”

Example 1.1 Applying a Model

Research question: How may the rapid increase in Medicare costs (Medicare: federally funded health care for the elderly) be reduced?

Purpose: To recommend change(s) in Medicare coverage that will contain the costs of the Medicare program.

Procedure: An initial model is sketched out with the major variables representing general strategies to reduce costs. The research will identify and evaluate the basic types of strategies available.

Page 3 of 29Research Methods for Public Administrators

3/3/2021file:///C:/Users/Lamar%20Holmes/Desktop/VitalSource%20Bookshelf_%20Research%20Methods...

Discussion: The sketch places each variable in a separate “box.” The variables, or strategies, which also may be identified as “inputs,” are placed in boxes. Lines indicate the links between variables and arrows indicate the direction of the links. Here, each strategy was linked to the outcome of reducing the rate of increased costs. The arrows suggest that each strategy should reduce the rate of increase in Medicare costs. A sketch keeps track of the model’s variables and relationships, helps communicate the model to others, and facilitates discussion about the model and the research plan.

With the variables identified, the investigators may:

1 Identify feasible strategies associated with each variable. Should the number of beneficiaries be reduced by changing income requirements or age limits? What services should be reduced? How might less be paid for services?

2 Incorporate the specific strategies into the model. 3 Develop a research plan: What variables and relations will be studied first? Exactly how

will they be studied?

Knowing Which Variables to Include and How to Link Them

To decide what to include, the researcher draws on her own ideas and experiences and the ideas and experiences of colleagues. She will usually review previous research and analyze data that have already been collected on the topic and will often conduct a preliminary study. In the Medicare example, she may consider the number of procedures covered and the rate of increase is the number of people eligible.

Before we study model building in depth, let’s look further at the preliminary model designed to guide the Medicare research project. Investigators identified the variables representing the general strategies available to reduce costs. The central variable was the rate of increase of Medicare costs.

Page 4 of 29Research Methods for Public Administrators

3/3/2021file:///C:/Users/Lamar%20Holmes/Desktop/VitalSource%20Bookshelf_%20Research%20Methods...

The other variables were the number of beneficiaries, the number of services covered, and the payment for each service; these variables were linked to Medicare costs in this model. If the number of beneficiaries, the number of covered services, or the amount paid for each service were reduced, the rate of increase in Medicare costs would also decrease.

Example 1.1 presents the model as a schematic or diagram. The arrows link the variables and show those thought to act on other variables. A model also may be presented as a statement. Another way to present the model would be with a sentence: “If the number of beneficiaries, the number of covered services, or the amount paid for each service were reduced, Medicare costs would increase more slowly.” A model also may be presented as an equation as shown below:

Rate of increase in Medicare costs = number of services covered + amount paid for each service + number of beneficiaries

The investigators assumed that reducing the number of beneficiaries, the number of covered services, or the amount paid for each service would reduce the rate of increase in Medicare costs. The major relationships are between reducing the rate of increase and the other variables; however, the other variables may be related to each other as well. For example, reducing the number of services covered may very well reduce the number of beneficiaries. The model could then show a link between these two variables.

Organizing and Refining Your Thoughts

Building a model requires the researcher to organize his or her thoughts and to communicate them effectively to others. The model in Example 1.1 should center policy makers’ attention on possible solutions and facilitate their communications among themselves and others concerned with Medicare costs. With the preliminary model sketched, investigators can move on to defining precisely what they mean by each variable. They may also decide to include additional variables and explore the linkages among them. They can decide which linkages to investigate further and with what priority.

A model may be altered and refined often during the early stages of designing a study. Many variables may be listed and linked, or only the essential components may be included.

How detailed should a model be? It depends. Investigators may brainstorm to identify all possible variables, and they may build a very detailed model. To confirm points of agreement, they may develop a simplified version with just a few variables included. The model in Example 1.1 was limited to the basic components of a study. Other models may elaborate on the basic model. For example, a model can spell out how specific strategies for reducing the number of beneficiaries would affect Medicare rates. Another model can focus on ways of reducing services and the effects of alternate strategies on Medicare rates. Regardless of the purpose or level of detail, a research model should be based on theory. That is, we should have good reasons to expect the linkages between the variables that our model posits and be able to explain them to others.

In addition, you should think about the assumptions underlying the theory. For example, if you are planning to do research on the relationship between automatic seat belts in cars and vehicle accident deaths, theory would suggest that the greater the percentage of cars having automatic seat belts, the lower the number of vehicle accident deaths. However, this expectation assumes that automatic seat belts are actually used and not disabled or turned off.2

Page 5 of 29Research Methods for Public Administrators

3/3/2021file:///C:/Users/Lamar%20Holmes/Desktop/VitalSource%20Bookshelf_%20Research%20Methods...

Maps provide a simple illustration of how models vary according to their purpose. For a mountain hike, you need a topographical map with details about the terrain. Such a detailed map, if you could find one and fit it into a car, would be virtually useless on a drive from New York to California. On the other hand, imagine using an ordinary road map to hike through the Grand Canyon! Variables important for one purpose may not be useful for another. And just as developing a research model is based on good theory, choosing a map is based on understanding where you want to go and how you think you might get there.

Consider the Model to Be Preliminary

The model should be considered preliminary. It may change during the course of a study. Model building starts the researcher on an iterative process to collect, analyze, and present data consistent with the study’s purpose. Models allow us to go beyond simply looking at data. They help identify why relationships are expected. Precise-looking data can be wrong, and apparent relationships among variables may be due to statistical errors. By the end of a study, models enable users to organize their information and reach reasonable conclusions about the importance of variables and their relationships to one another.

Even though models rarely stay the same throughout a study, a researcher should not begin collecting data without an explicit model and should be reasonably satisfied that this model includes all relevant variables. The included variables and their relationships become important components of decisions about which data to collect and how to analyze them. Data collection and data analysis often require costly amounts of time, expertise, and funding. Without a proper road map, researchers can wander in the wilderness of data—gathering numbers and other pieces of information without a clear direction. Without a proper map, a researcher risks collecting data that are not needed and not collecting data that are necessary.

Try to avoid “falling in love” with your models. This happens when a person has labored over a research plan and then becomes trapped by his own inflexibility. Instead of considering the model as preliminary and rejecting it or adjusting it as appropriate, the researcher goes through all sorts of data manipulations to demonstrate that the preliminary model is correct. The model should not be the focus of your work. The focus should be having the best information for the right people in time for them to make an informed, considered decision.

Building the Model The order associated with research presentations springs from a desire and need to communicate effectively. An audience can more easily follow a presentation that proceeds from a statement of the problem to a description of the model, its variables, and their relation to each other. Normally, a researcher does not report on false starts, mistakes, and backtracking when presenting the results of a study.

Yet the process of developing a research project is seldom as systematic and logical as textbooks, research reports, or public presentations make it appear.3 Designing a study requires creativity and insight. Nor does each investigator follow the same strategy for identifying variables and their relationships. In general however, to identify the variables, defend their relevance to the model, and postulate the nature of their relationships, investigators integrate their own ideas and research experiences, specific knowledge of the topic, the observations of others, and the existing research

Page 6 of 29Research Methods for Public Administrators

3/3/2021file:///C:/Users/Lamar%20Holmes/Desktop/VitalSource%20Bookshelf_%20Research%20Methods...

literature. Furthermore, researchers must invest time in understanding the specific problem at hand in order to engage in good model-building. This section discusses this process.

Steps in Conducting a Research Project

Research methods textbooks typically present a format for conducting research projects. Course instructors include similar guidelines in course material. These formats are helpful instructions for carrying out the research. Beginning researchers should keep in mind, however, that although all steps in such formats usually must be completed and generally in the order presented, in doing their work researchers often go back and forth among these steps. They may work on more than one step at the same time. Even so, experienced and beginning researchers should be familiar with the following steps. They need to understand and complete each step before the research project is complete.

Conceptualizing

The following questions should be answered at the beginning of the study. In most projects, answers to many of these questions will be included in the early part of the final research report.

What are the purposes and objectives of the study? What is the research question? To what cases are the results to apply?

It is common to address these issues in a section titled “Problem Statement.”

Literature Review

Review what others have written about the selected topic. From the review the researcher should learn about theories and hypotheses used by others, about data collection techniques and about ways of measuring variables.

Research Objectives and Hypotheses

Typically researchers will have hypotheses and sometimes objectives and hypotheses. Hypotheses should clearly state a relationship between two variables and be listed separately. The researcher should explain why each is likely to be supported. Variables are also identified and defined in this section. Variables must be identified as independent or dependent. Possible control variables should also be discussed.

Research Design and Data Collection Method

This includes the plan for how the research will be done: what data will be necessary, how it will be collected or what data source(s) will be used, how variables will be measured (operationalized), and sampling procedure. (Some of this information may appear in the previous section, especially with regard to variable definitions.) The cases or units of analysis for the study must be identified—these are the subjects about which data will be collected.

The method of collecting data must be described. From whom will the data be collected and how often? Will this be an experiment, a survey, document research? The specific technique should be discussed in detail.

Describe the research population. Will a sample be selected? If so the sampling procedure must be described.

Page 7 of 29Research Methods for Public Administrators

3/3/2021file:///C:/Users/Lamar%20Holmes/Desktop/VitalSource%20Bookshelf_%20Research%20Methods...

Data Analysis and Interpretation

The data are analyzed to test the hypotheses. The nature and strength of the relationships between variables should be described, and the researcher must clearly explain why the hypotheses are supported or not supported.

Implications

In addition to analyzing the data and testing the hypotheses, the researcher should explain what the results mean. How do they contribute to a body of literature, analysis of a policy, or solving of a problem?

Reporting

The researcher writes a report. In doing so, the researcher must keep in mind the audience for the research.

Ideas

Ideas refer to the knowledge, beliefs, or impressions one has about a research question. We rarely approach a situation with no knowledge or insight. Most of us retain a wealth of information to help us solve problems. Do not downplay the value or importance of this information. If you do not make use of your experience, knowledge, or opinions when faced with problems, you will not be efficient as an administrator.

Building a model requires you to formalize your existing ideas by requiring you to make your ideas explicit and communicable to other people. Through words, drawings, or equations, you identify the variables you consider important and how you believe they are related to each other. Thus you clarify fuzzy ideas and expose them to critical examination. You will find that some of your beliefs will not make sense. Some of your ideas suddenly may seem naive or incomplete. Some will be based on assumptions with which others disagree. Nevertheless, unless you are willing to take the risk of having your ideas challenged, you may miss variables and relationships critical to problem solving.

Peer Interaction

Peer interaction refers to the discussions and debates among colleagues about the research question and possible answers. To understand model building, we recommend that you read about how scientists go about their work.4 Doing so will quickly dispel the myth of the isolated scientist working alone. A major lesson is the value of criticism and debate. Without it, errors may go uncorrected and oversights may abound.`Few people find criticism easy to accept or arguments easy to verbalize, but the process of peer discussion about a research project uncovers assumptions and blind-spots.

A particularly important reason for peer interaction in public administration is that our professional and educational backgrounds tend to shape our ideas in ways that we may not recognize. For example, our disciplines shape the questions we ask, how we approach problems, the data we use, and how we use them. Imagine, as a city administrator, you are interested in studying ways to address homelessness in your city. As a first step, you decide to gather data on the causes of homelessness in your area. One analyst with an economics background may want to focus on quantitative data from city employment and support programs. Another analyst with a background in social work may turn

Page 8 of 29Research Methods for Public Administrators

3/3/2021file:///C:/Users/Lamar%20Holmes/Desktop/VitalSource%20Bookshelf_%20Research%20Methods...

first to interviews with clients of local shelters to learn their lived experiences. An analyst with a public safety background may wish to test a possible relationship between homelessness and parolees from prisons in the area. If they had to work together, we suspect they might experience frustration and tension. Yet, ideally, the conflict of ideas and the resulting compromises should yield a research plan superior to that developed by any one of the individuals working alone. One name for the concept of drawing from other disciplines is “borrowing strength.”

Review of Existing Knowledge

Research often is an iterative process in which investigators build on the work of others. Shortly after identifying the purpose of a study, investigators search for information on similar studies. Some people assume that quantitative researchers have little need to do this. Applied researchers may not see their studies as part of a larger body of knowledge. Nevertheless, a few hours reviewing academic journals and reference documents should convince you of the value of published materials whether you are conducting quantitative or qualitative, academic or applied research.5

Too often researchers studying an unfamiliar problem start by “reinventing the wheel.” In other words, they may begin to build a model that has already been developed and refined many times. The recommended approach is to first conduct a literature review, locating and examining what has already been written on the topic. The objective is not to ensure the originality of ideas. Rather, it is to identify information and ideas that we may incorporate into our own research. As we conduct the literature review, we think about the planned study and better understand its purpose and what we can expect to accomplish. An examination of previous research identifies the following:

 models used by other investigators  variables included in studies with a similar purpose  definitions currently used for the variables  techniques for measuring the variables  sources of data  strategies for collecting data  theories used to link the variables  the strength of the relationships between variables  results and conclusions  suggestions for further research

With this information, investigators avoid wasting time. They may find that their research question has already been answered. They may learn that others failed to confirm relationships that at first seemed important or obvious.

“State-of-the-art” articles that summarize the nature of existing research are especially helpful. They cite the major research work in the field and the major research themes. You will find that such articles quickly bring you up to date. They should have covered the relevant literature, easing your burden. They may also help you avoid the temptation to go too far afield in your investigation.

You will find that a reference librarian is an invaluable resource in locating appropriate literature. Most article abstracts are now available electronically and accessible online. A university reference librarian can introduce you to these and other useful online resources and suggest efficient search strategies for locating relevant research. He or she can show you how to use appropriate search

Page 9 of 29Research Methods for Public Administrators

3/3/2021file:///C:/Users/Lamar%20Holmes/Desktop/VitalSource%20Bookshelf_%20Research%20Methods...

engines, some of which focus on particular scholarly fields. Conducting online research can be both valuable and frustrating. Search engines can uncover a wealth of information or a pile of useless or, worse yet, erroneous information. One must be careful about depending on poorly sourced material. The most useful literature is published in peer-reviewed academic journals and books. The peer- review process ensures that experts have reviewed the articles and judged that the research employed and correctly carried out appropriate techniques of data collection and analysis. Other sources, such as trade journals, may also be helpful. However, keep in mind that the information in them typically does not undergo the same peer-review process.

Bringing Information Together to Build a Model

Example 1.2 suggests how an investigator brings together ideas, peer interaction, and the existing literature to build a model. The example applies model building to the process of conducting citizen surveys. If given the opportunity to build an original survey, you should develop the model that you plan to use before developing the survey in order to ensure that the survey will obtain needed information. Like many investigators, you may be inclined to design a questionnaire. Or you may plan to adapt one that has been used before. If done before explicitly building a model, however, you may find that the questionnaire or questions are not appropriate for your study. You may not be able to examine explanatory research questions using the resulting data. Try to avoid the temptation to skip the model-building step.

Example 1.2 Building a Model

Research questions: What do residents think about their town? Are they satisfied with available services? What changes do they want?

Purpose: To consider citizen perceptions in preparing the town’s comprehensive plan. Procedure: The planning director planned to survey residents to learn how they judge

the town and its services.

To identify variables of interest she

1 solicited existing surveys from local planners (peer interaction) 2 examined handbooks published by the American Planning Association and the

International City/County Management Association to find sample surveys (literature review)

3 drew on what she learned during her professional training, from her career experience (ideas), and from shop talk (peer interaction)

Several variables seem linked to perceptions of public services. She selected the following variables as relevant:

1 where residents live; if they are homeowners or renters (based on the town council’s interest)

Page 10 of 29Research Methods for Public Administrators

3/3/2021file:///C:/Users/Lamar%20Holmes/Desktop/VitalSource%20Bookshelf_%20Research%20Methods...

2 the gender and age of residents; whether they work in town or in the nearest city (based on the planning staff’s interest)

She met with the Planning Advisory Board to consider what services to include. They discussed the value of linking the demographic variables to satisfaction with services.

Summary of model: The survey will gather data on citizen satisfaction with public services, including trash collection, medical facilities, and housing. To see if the town government meets the needs of all residents, the planning staff will examine whether residents with different characteristics (gender, age, location of home or job, whether they rent or own) rate services differently.

To organize her presentations, she sketched the model:

Discussion: To develop the survey, the director had to decide which variables were relevant for the town’s planning. To do this she used her own ideas, peer interaction, and the literature. She met with her staff, the manager’s staff, an advisory board, and

Page 11 of 29Research Methods for Public Administrators

3/3/2021file:///C:/Users/Lamar%20Holmes/Desktop/VitalSource%20Bookshelf_%20Research%20Methods...

the town council to review and refine the survey’s content. The director read the literature to identify possible survey questions and to get ideas on how to construct the sample, how to achieve a higher response rate, and how to analyze the data. Next, she will scrutinize the specific questions, pretest the questionnaire, and pilot the study. As she conducts these steps, she may make some modifications to the preliminary model.

Examining a Dataset

Investigators with access to existing datasets, that is, data from previous studies, may examine and analyze some of that data as they focus and develop their ideas. In this way investigators may get a better idea of the importance of variables and the strength of their relationships.6 If a dataset does not contain the specific variables desired, the investigator may study similar variables, which can be considered as stand-ins or proxies.

For example, a neighborhood or postal ZIP code may act as a proxy measure for racial or ethnic groups, social class, or even family composition. Or the researcher may decide that a dataset does not offer an acceptable way to capture one or more variables that are part of the planned model. The number of datasets available online and the variety of topics covered is immense. Many are easy to access.

Investigators manipulating a dataset may rethink their study’s purpose and their model’s adequacy. Through the process they may gain greater insight into their own study and revise their original ideas.

Pilot Testing the Model

Before collecting data on a large scale, the research plan should be rehearsed. A small study, called a pilot study, is launched to test the adequacy of the proposed data-collection strategy. During the pilot study, investigators discover the feasibility of their research plans and how much time and effort will be needed to collect, compile, and analyze the data. In the pilot study, investigators should carry out the entire plan, including the data analysis and interpretation of results. Often investigators fail to analyze and interpret the pilot test data. This is unfortunate, as they then miss opportunities to test the model’s appropriateness, identify important variables not included in the original model, or find that some variables in the model are not needed.

Collecting data without a preliminary model is not a sound research practice. Too often research projects begin prematurely. After identifying the research question, some investigators begin collecting data eagerly. They cut off further planning with comments such as, “Let’s see what the data show,” or, “I’ll decide after I look at the data.” Questionnaires are constructed, subjects are questioned, and data are analyzed with too little attention given to the investigators’ objectives. Unfortunately, the investigators may then not recognize problems until the end of the research project.

Types of Models At this point you may be wondering: “How do investigators present a model’s variables and relationships?” While many possibilities exist, ranging from physical models, such as mock-ups of buildings, to highly abstract verbal presentations, administrative researchers usually work with two

Page 12 of 29Research Methods for Public Administrators

3/3/2021file:///C:/Users/Lamar%20Holmes/Desktop/VitalSource%20Bookshelf_%20Research%20Methods...

types of models: schematic models and symbolic models. Schematic models refer to models that use pictures, lines, points, and similar paper-and-pencil type products to designate the variables and illustrate their relationship to each other. The sketches in Examples 1.1 and Example 1.2 present the model as a schematic, as do blueprints, flowcharts, and maps. Symbolic models refer to models that use words or equations to represent the elements and describe their relationships.

The type of model chosen depends on the model’s purpose, the audience, and the investigator. More than one type of model may be used to present a set of variables and relationships. Consider getting directions to a person’s house. Some people distribute maps—a schematic model. Others give verbal directions, listing roads, landmarks, and distances—a symbolic model. Others may give both a map and verbal directions. The choice depends largely on a host’s preferences and perceptions of what will be easiest for his guests.

Many investigators build both a symbolic and a schematic model, as is done in the Example 1.2, “Building a Model.” The example’s symbolic model summarizes the planning director’s actual verbal model, which establishes the importance of linking each specific demographic characteristic to the residents’ perception of community services. The schematic model sacrifices the detail of the symbolic model, but it effectively identifies the model’s essential features.

Schematic Models

Schematic models work well in summarizing the thinking behind the model and in drawing attention to its major features. Schematic models with a few variables can be understood quickly and focus people’s attention. Consequently, schematics help an investigator think through the model and explain it to others. A schematic also may serve as an elaborate checklist. For example, an investigator can record what data are needed to measure a variable and when those data are gathered or when a relationship has been examined. Schematic models are less effective if they include too much detail. Only so much information can be included about a variable and its relationship to other variables before a schematic becomes cluttered and confusing.

Most administrators and students of public and nonprofit administration are familiar with schematic models such as flow charts and logic models. Logic models illustrate the components in program development and operation. Logic models use a standard format and components are fitted to this format for specific programs.7

Figure 1.1 shows the standard form of a logic model with an illustration of a housing program.

Symbolic Models

Symbolic models include verbal and mathematical models. Verbal models use words to describe the variables and define their relations to each other. Verbal models are often found in the introductory or theoretical sections of research articles. A newspaper account of a research study and its findings also may be considered a verbal model.

This type of model has distinct strengths. The investigators have all the potential offered by language to describe the model, allowing a full and detailed explanation of complex relationships. The researchers are not constrained by requirements to reduce the model to equations. A fuller range of users can understand and interpret a verbal model than can work with mathematical models. Verbal and mathematical models can complement each other in describing relationships. The verbal model provides the rich details, and the mathematical model looks at the precise nature of the relationships.

Page 13 of 29Research Methods for Public Administrators

3/3/2021file:///C:/Users/Lamar%20Holmes/Desktop/VitalSource%20Bookshelf_%20Research%20Methods...

Figure 1.1 A Logic Model Applied to a Housing Program Source: Adapted from Figure 4.2 in O’Sullivan, Rassel, Taliaferro, “Practical Research Methods for Nonprofit and Public Administration.” Longman-Pearson, 2011

Mathematical models can condense a great deal of information by using equations to specify relationships. The mathematical models indicate which relationships exist or are expected, indicate their directions, and measure their strengths. Mathematical models allow administrators to predict needs, to estimate the impact of policy decisions, and to allocate resources more efficiently. In some areas of interest to administrators, such as operations research or management science, mathematical models guide the model-building process.

Complicated mathematical models often are handled best with computers. In such instances a software program can quickly manipulate the variables and depict their relationships statistically and graphically. Unlike an analyst, the computer can keep track of a large number of variables and the relationships among them. A computer model differs from other models primarily in the amount of information it can manipulate, its speed, and its accuracy. It is less likely to make recording or computational errors. As with other models, however, the accuracy of a computer model is a function of human judgment that creates the rules for selecting and manipulating variables. No matter how many variables are included the model still represents a simplification of reality.

Limitations of Models and Model Building for Specific Users No matter how many variables are included, the models still represent a simplification of reality. They represent one view of which variables are relevant to the problem at hand and which variables can be ignored. Differing views may be entertained. The investigators who build a model and the people who critique it are subject to the full range of human weaknesses. Model builders contend with limited time, money, and knowledge. Their viewpoint may be colored by biases that lead them to ignore others’ comments and criticisms.

Users may be seduced by a model’s clarity and its apparent usefulness. They may assume erroneously that it is accurate and adequate. Users who read a study or sit in an audience tend to focus on the details of the verbal description, a diagram, or an equation. They may want to step back and reflect on what the model leaves out.

Researchers who initiate a study at the request of a policy maker or administrator may have a similar concern. The public administrator or policy maker who requests the study should discuss its audience and use with the research team and should also ask the research team to explain the logic of its model in clear terms. For full-time researchers, models serve as a logical mechanism to organize their thoughts and studies, but this is not necessarily the case for public administrators as “clients.” If a client must adapt to the investigator’s methods, the study may be logical and well organized, but it may be ill suited to the client’s needs. The research team should not force the administrator or policy maker who requested the study to conform to its own way of approaching a problem.

Page 14 of 29Research Methods for Public Administrators

3/3/2021file:///C:/Users/Lamar%20Holmes/Desktop/VitalSource%20Bookshelf_%20Research%20Methods...

Understanding Users’ Goals

In conducting research for specific users, the investigator works hard to understand why the study is being done and what the users want to do with the findings. Conversations between administrators and the investigators may seem far removed from identifying the model’s variables and relationships. Nevertheless, the research study’s clients have valuable perspectives on potential models. A researcher also has to deal with the reality of trying to satisfy client demands or individual situations. For example, to evaluate strategies to have toddlers vaccinated, the analyst may ask the client to speculate on which factors motivate parents to do so and which factors lead them to reject or to delay vaccinations. The analyst will then organize the information into a symbolic or schematic model.

The researcher may leave many of the research details in the background. A client may receive a list of research questions with a strategy for answering them or a description of an experiment. A user might review graphs and tables similar to those planned for the final report. This strategy mirrors the model-building process. Beginning with the clients’ perspective avoids the mistake of ignoring the observations and concerns of users who cannot articulate their ideas within the context of an explicit model. Focusing on the clients’ concerns and needs ensures that a study will be planned to address a specific problem. Otherwise analysts may plan a study around their own analytical preferences. Involving the client in the model building process may help the client identify the true purpose of the study.

In conducting research to answer theoretical questions, such as the nature of organizational leadership, investigators pursue model building as described earlier. They include peer interaction, a literature review, and pilot-test information in building and refining their models. If they report their findings in a scholarly journal, they will use the shorthand afforded by research jargon. In theoretical research, a study evolves from the research that precedes it. Theoretical studies develop a discipline’s body of knowledge and set the stage for further empirical research. Consequently, the researchers must provide a detailed discussion of their methodology and conduct extensive statistical analysis of quantitative data.

Applied studies generally are done with intended users in mind. In these, the literature review may be less thorough and systematic and more narrowly focused. Specific users combine research findings with other information to guide their thinking about a problem or to make a decision. Investigators avoid research jargon in communicating with most administrators and policy makers. The jargon serves as a checklist for the researcher, so he does not overlook the numerous details that can weaken a study, but he avoids using jargon otherwise. If the terms are unfamiliar to audiences, their attention may shift to understanding the words, and they may ignore the important points that the investigator wants to emphasize. This point is valuable as you read this chapter’s next section. We introduce a wealth of terminology, but if you read a report, you may never see these specific terms. Nevertheless, in any report, you still should find ample evidence that the researchers built models and applied standard research procedures in designing their study.

The Components of Models The primary components of models are variables. Variables are observable characteristics that can have more than one value, that is to say, characteristics that vary. Some examples of variables and their values are shown in Table 1.1.

Page 15 of 29Research Methods for Public Administrators

3/3/2021file:///C:/Users/Lamar%20Holmes/Desktop/VitalSource%20Bookshelf_%20Research%20Methods...

If the characteristic has only one value in a study, it does not vary and is called a constant. For example, researchers will often use as a study population those cases with only one value of a characteristic. For example, in a study of the relationship between employee age and preference for team projects, the researcher included only men. In this study, gender was a constant. Another constant would have been type of industry—only one manufacturing industry was included.

Table 1.1 Variables and Variable Values

Variable Values Gender Male, Female Job satisfaction Very satisfied, Satisfied, Dissatisfied, Very dissatisfied Salary Actual dollar amount of salary Age Number of years since birth

Hypotheses

To test a model, researchers examine the relationship between variables linked in the model. The investigators express the relationship between two variables in a simple model called a hypothesis. A hypothesis is a statement that specifies or describes the expected relationship between two variables in such a way that the relationship can be tested empirically. Hypotheses form the foundation of a research effort. A clearly written hypothesis helps researchers to decide what data to collect and how to analyze them.

Consider the following four examples of hypotheses. Can you identify the variables in each hypothesis?

H1: Persons jailed for burglary are more likely to be rearrested once released than are those people jailed for assault.

In this hypothesis, the variables are the type of crime for which someone is jailed and the probability of being arrested again.

H2: City governments keep more supplies in storage than do county governments.

The variables in the second hypothesis are the type of government and the amount of supplies kept in storage.

Note that the hypotheses H1and H2 are reasonably specific. We have a good idea of what data to collect to measure the type of crime, whether a person arrested has been arrested before, type of government, and the amount of supplies kept in storage.

H3: Training programs improve the skills of the chronically unemployed.

Now in this hypothesis, imagine the frustration of the researcher trying to test it. And yet it is a reasonable beginning point for a research project. It is just not a useful hypothesis since it is not easily testable as stated here. To what type of training program does the hypothesis refer, and what does its author mean by “skills”? We may even debate who qualifies as chronically unemployed. The vagueness of the hypothesis leads us to suspect that its author has a poorly developed model.

Page 16 of 29Research Methods for Public Administrators

3/3/2021file:///C:/Users/Lamar%20Holmes/Desktop/VitalSource%20Bookshelf_%20Research%20Methods...

H4: The city manager form of government is more common in the Southern states.

In Hypothesis 4, a value is given for independent and dependent variables; however, the variables are only implied. They are “region of the country” for the independent variable and “form of city government” for the dependent variable.

The decision on how specific a hypothesis should be is relative. Some variables, such as skills and who is chronically unemployed, can be defined as the researcher decides how to measure a variable and from whom to collect data. Even apparently specific variables may need further definition. Consider the crimes of burglary and assault. Each of these has several categories, from less to more serious. What types of supplies should be included in collecting data to test the second hypothesis?

Two Kinds of Variables

The most useful hypotheses typically include two variables, an independent and a dependent variable, and imply that a change in one variable is caused by a change in the other variable.

• The dependent variable is the characteristic whose changes the researcher wants to explain. The independent variable is considered the explanatory or causal variable.

• The independent variable is used to explain variation in the characteristic or event of interest. It is sometimes referred to as an “input” or “causal variable.” The dependent variable represents or measures the characteristic or event being explained. It also is referred to as an “outcome” or an “effect.”

One may visually identify the independent and dependent variables with a schematic model; the arrow leads from the independent variable to the dependent variable. Another way to think about dependent and independent variables is to ask, “Which variable depends on the other?” The goal of research often is to identify the causes of problems or social conditions we wish to change. To confidently claim that one variable is really the cause of another, however, requires four types of evidence. Good hypotheses usually state a relationship between or among variables, and researchers test for those relationships. That two variables are related does not mean that one causes the other.

Some people find it helpful to rephrase a hypothesis as an “if-then” statement. For example, “If the age for Medicare eligibility is raised, then the rate of increase in Medicare costs will be slowed.” The “if” statement contains the independent variable, that is, age for Medicare eligibility. The “then” statement contains the dependent variable, that is, rate of increase in Medicare costs. In our example hypotheses, the independent variables were category of crime, type of local government, training programs, and region of the country. The respective dependent variables were the probability of being arrested again, the amount of supplies in storage, and quality of life.

We have defined hypotheses as consisting of an independent and a dependent variable. Consider a hypothesis with two independent variables. For example, “women and older adults use public libraries more often than men and younger adults.” Is it necessary that both women and older adults use public libraries more for the hypothesis to be supported? What if women and younger adults more often use public libraries? Or does the independent variable have four values: older women, younger women, older men, and younger men? If this is the case, the hypothesis should make the values explicit. Otherwise, to avoid the ambiguity of having one part of a hypothesis supported and the other part unsupported, it is conventional to have one hypothesis for each independent variable. In our example, the hypotheses would be:

Page 17 of 29Research Methods for Public Administrators

3/3/2021file:///C:/Users/Lamar%20Holmes/Desktop/VitalSource%20Bookshelf_%20Research%20Methods...

1 Women use public libraries more often than men. 2 Older adults use public libraries more often than younger adults.

Characteristics of Good Hypotheses

The most useful hypotheses state a relationship between two or more variables. However, we can also state and test hypotheses containing only one variable. For example, a state demographer may hypothesize that “the population of the state’s capital city is over 250,000.” Or the director of economic development for a resort community may claim that summer concerts bring $2 million of economic activity to the community. The only variables in these hypotheses are “size of population” for the first and “number of dollars of economic activity” for the second. Testing such hypotheses can provide important information. However, testing hypotheses relating two variables helps us understand why the dependent variable varies. Explanatory research questions always involve testing hypotheses that have both a dependent variable and an independent variable.

Good hypotheses have the following characteristics:

• They include one independent and one dependent variable. • Each variable is clearly stated. • From the wording it is clear how each variable varies. • The expected relationship between the variables is clearly stated.

Units of Analysis

When testing hypotheses the units of analysis used in the research should be at the same level of aggregation as those in the hypothesis. Units of analysis are the cases or entities for which we measure variables. For example, the units of analysis in the Belle County Data file (Belleco) or (Belleco.sav) are individuals who responded to the survey. The units of analysis in the County Data file are counties. Many of the county variables are summaries of items in each county. The percent of people with a college degree, for example, is a county-level variable. It is based on counting the number of people in the county with a college degree. The County variables, therefore, are at a higher level of aggregation than the Belle County variables.

Consider the following example of an error that could occur if the units of analysis for the data are not at the same level as those in the hypothesis. Assume that a researcher is interested in factors related to student SAT scores. He states the following hypothesis:

“The better the student’s school attendance record the higher the student will score on the SAT exam.” (Note that the units of analysis are individual students. School attendance is the percent of total school days attended.)

The researcher does not have data on individual students but does have data from a large number of school districts. He tests the hypothesis and finds that the average SAT scores of students from school districts with higher average student attendance is higher than average SAT scores of students from districts with lower average attendance. This researcher cannot then conclude that individuals with better attendance records scored higher on the SAT exam. They may have; however, since the data used to test the hypothesis were based on school districts, the researcher cannot assume that the results also apply to individual students.8

Page 18 of 29Research Methods for Public Administrators

3/3/2021file:///C:/Users/Lamar%20Holmes/Desktop/VitalSource%20Bookshelf_%20Research%20Methods...

The Nature of Relationships

An important characteristic of a hypothesis is the pattern of the relationship it postulates. Covariation refers to the patterned relationship between an independent and a dependent variable. Covariation between two variables commonly takes one of three forms: positive—also called direct, and negative—also called inverse, or nonlinear.

To describe the patterns of covariation, consider the relationship between the amount of job training received and salary on the first job after training. The relationship may be positive; as the number of training hours received increases, the amount of salary increases. The relationship may be inverse, or negative; as the number of training hours increases, the amount of salary decreases.

The relationship may be nonlinear in that a distinctive but nonlinear pattern occurs. For example, two such patterns might emerge from a study of the relationship between training and salary levels. In the first, salary increases as the number of training hours increases, but only to a point, beyond which it begins to level off or decrease as training increases further. In the second, as the amount of training increases, salary increases to a point, after which the amount of salary received stays constant. Figure 1.2 shows examples of positive, negative, and nonlinear relationships.

If the independent variable has no discernible effect on the dependent variable, we say that the two variables do not vary together. If two variables do not vary together, their relationship may be described as random or null.

The Role of Control Variables

In stating and testing a hypothesis, we may wonder about the effects of other variables on a hypothesized relationship. Researchers may suspect a variable not included in the hypothesis is related to the independent and dependent variables and affects the relationship between them. To account for this, they add that variable to the analysis to see if it alters the relationship between the independent and dependent variables. Such a variable is called a control variable because including it in the analysis “controls for” any affect it has on the relationship between the independent and dependent variables. Many researchers use the term independent variable for both the core independent variable of interest in the hypothesis and the control variables. As you build a model and work on a literature review, keep in mind a distinction between the independent variable centrally involved in your hypothesis and the independent variables less central to your study as control variables. For this reason, some researchers call the central independent variable the explanatory variable and all other independent variables control variables.

Page 19 of 29Research Methods for Public Administrators

3/3/2021file:///C:/Users/Lamar%20Holmes/Desktop/VitalSource%20Bookshelf_%20Research%20Methods...

A control variable may show that the original relationship was in error, e.g., the relationship was false or spurious.

The following examples illustrate spurious relationships. First, consider the hypothesis “arrests for assaults increase with ice cream sales.” In other words

these two variables are positively related. If subsequent research shows a statistical relationship, should you conclude that ice cream causes physical violence? Of course not; average daily temperatures (a control variable) may reveal that hot weather, not ice cream sales (the independent variable), is associated with more assault arrests (the dependent variable). Ice cream sales typically are higher during hot weather than during cold weather; so is the number of assaults.

Similarly, hospital administrators challenge reports that their hospitals have a higher than expected death rate. They contend that their hospitals treat sicker patients. Thus they argue that patient prognosis at admission (a control variable), not the hospital itself (the independent variable), causes the higher death rate (the dependent variable).

Moderating Variables

In addition to helping identify a spurious relationship, control variables may alter the hypothesized relationship radically. Such variables are called moderating variables—they moderate or change the relationship between the original two variables. Consider how noise affects productivity. If a task requires concentration, noise tends to diminish productivity, whereas if a task is monotonous, noise tends to increase productivity because it jolts workers out of their daydreams. In Example 1.3, we show how a control variable is used to interpret the effectiveness of three job-training programs. When the relationship between the training program and student employment is considered, one program appears to be markedly more successful than the other two. When the educational level of trainees is considered, the difference becomes less striking. For students without a high school diploma or with education after high school, two training programs have nearly identical success in placing trainees.

Example 1.3 Illustrating Variables and Hypotheses

Problem: Identify successful job-training programs for hard-to-place individuals. Hypothesis: On-the-job training programs will be more successful than other training

programs in placing participants in permanent positions. Independent variable: Type of training program. Dependent variable: Placement in a permanent position. Control variable: Educational level. Findings: The data reported in the following list supported the hypothesis that on-the-

job training programs had the highest placement success. These data also showed that work-skills training had the lowest placement success.

Placement Success by Job-Training Program

Page 20 of 29Research Methods for Public Administrators

3/3/2021file:///C:/Users/Lamar%20Holmes/Desktop/VitalSource%20Bookshelf_%20Research%20Methods...

Vocational education program (attendance in a school-based training program): 31 percent of all participants placed.

On-the-job training: 41 percent of all participants placed. Work-skills training (program to teach basic reading and mathematical skills, attitudes,

and behavior needed for permanent, skilled employment): 23 percent of all participants placed.

Table 1.2 Placement Success by Program, Controlling for Years of Schooling (H.S. = High School)

Education Vocational Training On-the-Job Training Work-Skills Training <12 years 33% 35% 23% H.S. graduate 28% 43% 25% >12 years 38% 39% 25%

Discussion: The examination of the relationship between the independent and dependent variables for all participants shows that on-the-job training is markedly more successful in placing trainees (41 percent compared to 31 percent and 23 percent). If education level is controlled and held constant, vocational education and on-the-job training do about equally well in placing participants who did not graduate from high school and those who continued beyond high school. Based on these findings, a counselor would refer trainees who only completed high school to on-the- job training. Other participants may do equally well in either on-the-job training or vocational education. If you funded these programs, what decisions would you make? Would you want to look at other control variables? Which ones? Why?

Before you leave this example, make sure you can interpret the figures in the table with the control data. For example, 33 percent means that 33 percent of the people in vocational education who had not completed high school had been placed. And 43 percent of those in on the job training who had graduated high school had been placed. Researchers have different ways of presenting control data, so expect to take time to determine exactly what the reported data mean.

In the example of the relationship between noise and productivity, the relationship is not in the same direction for each value of the control variable. In the example of the relationship between job- training programs and student employment, the relationship is notably stronger for one value of the control variable than for the others.

What can you deduce from the introduction of a control variable?

1 The hypothesized relationship is spurious—it only appears to exist because the control variable is related to both independent and dependent variables, or

2 The relationship between the independent and dependent variables is stronger for some values of the control variable than for other values, does not have the same direction for each value of the control variable, or is supported for some values of the control variable and unsupported for other values, or

Page 21 of 29Research Methods for Public Administrators

3/3/2021file:///C:/Users/Lamar%20Holmes/Desktop/VitalSource%20Bookshelf_%20Research%20Methods...

3 The control variable has little or no impact on the relationship.

For studies of human behavior or attitudes, common control variables are age, sex, income, education, and race. For research on agencies, common control variables include agency size, mission or purpose, size of budget, and region of the country or state. As you read the literature about a particular policy or program, you may locate other control variables that are used regularly to understand the relationship between the original independent and dependent variables.

Selecting a Research Question Students may be overwhelmed by seemingly limitless possibilities of topics when required to write a research paper. Analysts face a similar situation as they decide how to study a policy and its impact or even which policies to study. Administrators have to motivate themselves to do research if it is not required in their jobs. The results of research studies, however, often provide important information. Students, analysts, and administrators all want to study relevant issues, that is, issues that engage public, professional, or agency attention. Recent relevant topics include public pension reform, teacher tenure, income inequality, the implications of changing demographics, immigration, and the consequences of rising sea levels. Other relevant issues can be gleaned from professional journals, professional meetings, and the national media. Researchers often choose topics by identifying problems discussed in these sources. A typical format for writing a research paper is to begin by briefly discussing the problem the research will address and why it is a problem.

Once a topic is selected, the researcher must focus his efforts in order to move on to the next steps in the process. Typically he does this by stating what he wants to find out as a research question. The research question can be framed by considering the questions we listed at the beginning of the chapter—questions that begin “how many,” “how much,” “how efficient,” “how effective,” “why,” and “what.” The research question should lead the researcher to develop hypotheses that will answer the question if tested.

Students’ Selection of a Topic

Students have the most freedom in selecting a topic. A required paper serves as an opportunity to study an unfamiliar topic or to develop new skills. Public administration students may seek projects that introduce them to agency officials or add to their resumes. In selecting a topic and framing the research question, students should do the following:

1 be interested in the topic 2 have, or be able to develop, the required knowledge and skills to conduct the study 3 have sufficient time and resources to conduct the study

A student may be encouraged to select a topic simply because it is “hot.” Unfortunately, a relevant topic has little value if the student cannot produce a credible study. Students may be hindered by a lack of interest in the topic or insufficient knowledge. For example, a review of the environmental policy literature may require knowledge of economics, statistics, or the physical or biological sciences—expertise that students rarely can develop within a semester. Students may underestimate the time and cost involved in empirical research—data collection and analysis take resources.

Page 22 of 29Research Methods for Public Administrators

3/3/2021file:///C:/Users/Lamar%20Holmes/Desktop/VitalSource%20Bookshelf_%20Research%20Methods...

Beginning and experienced researchers should keep in mind that a small, well-designed study is better than a large study that falls apart because of insufficient time or money.

Analysts’ Selection of a Topic

Analysts’ choices of research topics are more limited; topics must be relevant to the decision at hand. Similar to students, they should have requisite knowledge, sufficient time, and adequate resources; however, co-workers or contractors can supplement an analyst’s knowledge or skills. In selecting a topic, analysts consider its relevance to the agency and their careers. A relevant topic is one that is important to the agency’s mission, consumes substantial resources or affects the public’s quality of life, and is amenable to change. Employees with analytical skills are often assigned to study issues important to agency managers. Studies on inconsequential topics or on policy areas resistant to change are likely to be ignored. A record of producing unused studies is unlikely to lead to individual career or organizational success.9

Administrators’ Selection of a Topic

Administrators rely on research findings to identify problems, evaluate solutions, and make decisions. They use research findings to keep existing resources and to justify additional ones, to monitor programs, and to improve employee performance. An effective strategy for administrators in university towns or with strong alumni ties is to keep a list of research questions. The list is ready whenever a faculty member calls looking for a class project. Students get to work on a relevant project, and the agency benefits from a low-cost study. Even the most modest study teaches administrators more about a particular policy and its implementation.

Administrators may often rely on the knowledge and research skills of others. Nevertheless, as we stressed earlier, they should be sufficiently engaged to oversee the project and to question the research strategy and the findings. In our experience, working with researchers to define a research question, build a model, and critique the method and the findings contributes to administrators’ knowledge of their agency and their programs. In the end, the topic should matter to someone. Otherwise, there is no point in doing the research.

Summary Effective quantitative research requires that investigators articulate the purpose of a study. With the purpose in mind, the investigators can select the variables of interest and postulate their relationships to each other. These elements and relationships constitute a model. Investigators may find a model aids them in three ways. First, it helps them to explain their ideas to others and to solicit reactions and criticisms. Second, it helps investigators to understand their ideas better. Third, it provides a useful guide to the research. It is particularly valuable in ensuring that all specified variables are measured and analyzed.

In building a model, investigators consider their own ideas and the ideas of colleagues and review existing research. Models may also emerge as investigators work with existing data or examine the results of a pilot test. The specific way that investigators use these information sources varies from person to person and requires some creativity. Nevertheless, one can expect to rethink his or her ideas several times as information from different sources is brought to bear.

Page 23 of 29Research Methods for Public Administrators

3/3/2021file:///C:/Users/Lamar%20Holmes/Desktop/VitalSource%20Bookshelf_%20Research%20Methods...

In our opinion, the review of the literature is the most important source of information. A thorough review of the literature greatly reduces the likelihood of wasting resources, such as time and effort. It provides valuable information that can be used in the later stages of research planning. Frequently, it helps investigators to identify strategies they can copy or adapt. Investigators will want to make use of the resources of reference and documents libraries, particularly abstracts, statistical indices, citation indices, and the wealth of other electronic sources.

The preliminary model may be modified several times during the course of planning a study. Once a plan is developed, investigators should conduct a pilot test. It should include the analysis and interpretation of the findings. Pilot-test information can be reviewed by investigators and major study users to ensure that the final study fulfills its purpose.

Administrative researchers primarily use schematic and symbolic models. Schematic models help investigators to illustrate their models and explain them to others, but much of the detail that is needed to understand and test the model may be missing from the schematic. Symbolic models use words or equations to represent the variables and their relationships. Verbal and mathematical models often are complementary. The verbal model presents the model in all its detail; the mathematical model gives precise descriptions of the relationships between variables.

The components of models are variables, that is, characteristics that vary. An independent variable is an explanatory variable—it is often thought of as a cause. A dependent variable is the characteristic whose variation the researcher wants to explain—it is often thought of as the result. A hypothesis is a simple model that states a testable relationship between an independent and a dependent variable. Frequently, an investigator will add additional variables, called control variables, to see whether they alter the hypothesized relationship. Common control variables in studies of human behavior include race, gender, and age. In studies of organizations, number of employees, size of budget, and mission often are used as control variables.

Once the preliminary model has been outlined, the investigators must decide when and how often to collect data, how much control to exert over the study, what data to collect, and from whom to collect the data. To make these decisions they consider the purpose of a study, how it affects the timing and frequency of data collected, and the investigators’ degree of control over the research situation. A variety of approaches are available and used by administrators and researchers as sources of information. These approaches include cross-sectional studies, time-series analysis and case studies.

Notes

Page 24 of 29Research Methods for Public Administrators

3/3/2021file:///C:/Users/Lamar%20Holmes/Desktop/VitalSource%20Bookshelf_%20Research%20Methods...

1 See G. King, R. Keohane, and S. Verba, “The Importance of Research Design,” in Rethinking Social Inquiry: Diverse Tools, Shared Standards, 2nd ed., ed. E. Brady and D. Collier (Lanham, MD: Rowman and Littlefield, 2010), 181–192.

2 James Jaccard and Jacob Jacoby, Theory Construction and Model Building Skills: A Practical Guide for Social Scientists (New York, NY: The Guilford Press, 2010).

3 A. Kaplan, The Conduct of Inquiry: Methodology for Behavioral Science, 2nd ed. (New Brunswick, NJ: Transaction Publishers, 1998). Kaplan refers to this distinction as “logic in use”—how research is actually done—and “reconstructed logic”—how the textbooks and researchers tell us that research is done.

4 Extensive electronic resources on the scientific method are available. For an interesting look at science as a culture, see Bruno Latour, Science in Action: How to Follow Scientists and Engineers Through Society (Cambridge, MA: Harvard University Press, 1987). Another valuable text in this area is Paul Diesing, How Does Social Science Work? Reflections on Practice (Pittsburgh, PA: University of Pittsburgh Press, 1992). A continually evolving example of scientific method and practice is the research on DNA.

5 For information on writing a literature review, see: www.lib.ncsu.edu/tutorials/litreview; and J. Galvan, Guidelines for Writing Literature Reviews: A Guide for Students of the Behavioral Sciences, 4th ed. (Glendale, CA: Pyrczak Publishing, 2009). An Internet search will lead to additional guides.

6 For a discussion on empirical analysis as part of model building, see J. W. Tukey and M. B. Wilk, “Data Analysis and Statistics: Techniques and Applications,” in The Quantitative Analysis of Social Problems, ed. E. R. Tufte (Reading, MA: Addison-Wesley Publishing, 1970), 370–390. Although an older reference, this article is still valuable. The authors warn against exploring data with no model in mind and advocate that researchers not take their models too seriously or be unwilling to change them.

7 Chapter 3 in Wholey, Newcomer and Hatry. See specifically John A. McLaughlin and Gretchen B. Jordan, “Using Logic Models,” in Handbook of Practical Program Evaluation, 4th ed., ed. J. Wholey, K. Newcomer, and H. Hatry (San Francisco: John Wiley and Sons, 2010), Chapter 3, 55–80.

8 King, Keohane, and Verba, 30. The error involved in applying results from higher level cases to individuals is sometimes called an “ecological fallacy.” See S. Lieberson, Making It Count: The Improvement of Social Research and Theory (Berkley: University of California Press, 1985), and W. S. Robinson, “Ecological Correlations and the Behavior of Individuals,” American Sociological Review 15 (1950): 351–357.

9 See A. J. Meltsner’s “Problem Selection,” in Policy Analysis in the Bureaucracy (Berkeley: University of California Press, 1976), 81–113, for a provocative discussion on problem selection.

Terms for Review

models model building literature review pilot study schematic model symbolic model variables constants hypothesis values of a variable independent variable

Page 25 of 29Research Methods for Public Administrators

3/3/2021file:///C:/Users/Lamar%20Holmes/Desktop/VitalSource%20Bookshelf_%20Research%20Methods...

dependent variable unit of analysis covariation positive relationship inverse or negative relationship nonlinear relationship random or null relationship spurious relationship control variable

Questions for Review The following questions should indicate whether you have a basic competency in this chapter’s material.

1 Why should an investigator state a purpose for a model before building it? 2 Why should an investigator build an explicit, preliminary model before beginning a

quantitative study? 3 Identify the steps involved in model building, and comment on their importance. Select one

of the steps and consider the effect on the model if it were eliminated. 4 In general, would models built by one person be superior to a model developed by a group?

Justify your position. 5 a In the following hypotheses, identify the independent and dependent variables, two or

three possible values for each variable, and the direction of the relationship. Then suggest two control variables for each hypothesis:

 i Death rates in automobile accidents are higher in less densely populated areas. ii The higher the average driving speed on a highway, the higher the automobile

death rate on that highway. iii Defendants with records of alcohol abuse are more likely to miss scheduled

court appearances. iv Parents of elementary and high school students are more satisfied with their

children’s schools than are parents of junior high students.

b Revise hypotheses i and iii to improve them.

6 Develop three hypotheses, each with an independent and dependent variable on a topic of interest to you. Then suggest a control variable for each hypothesis. Create a schematic model including the control variable for each hypothesis.

7 Consider safety violations as an independent variable, and write three hypotheses. Indicate the direction of each hypothesis.

8 Using criteria discussed in Chapter 1, evaluate the following proposed “hypotheses”:

H1: School boards should not be appointed. H2: There are more African Americans than Hispanics on U.S. school boards.

Page 26 of 29Research Methods for Public Administrators

3/3/2021file:///C:/Users/Lamar%20Holmes/Desktop/VitalSource%20Bookshelf_%20Research%20Methods...

H3: Appointed school boards are more likely to have African American members than elected school boards.

H4: African American school board representation is measured by the percent of school board members who are African American.

9 Identify the independent variables and their values in the hypotheses:

H1: Older mothers with a college education are most likely to participate in parks and recreation programs; younger mothers with only a high school education are least likely to participate.

H2: Older mothers and more educated mothers are most likely to participate in parks and recreation programs.

What, if any, changes would you make to H1 or H2?

Problems for Homework and Discussion

1 Process models or flowcharts are often developed to describe and guide a process. One may use a process model to describe ways to build a model. Use a schematic to illustrate what you consider a “good” way to build a model. Show all linkages and their direction. You should have described either a systematic step-by-step process or a dynamic process in which the steps are repeated and the model is revised several times. Write a one page defense of the process you described.

2 Link the purposes of a literature review with the stages of model building. At what stage(s) of the model would you conduct a review, and what would be the purpose of the review at each stage?

3 Find a newspaper article that presents a verbal model.

a State the apparent purpose of the model. b Draw a schematic to illustrate the model. c State a hypothesis included in or implied by the model. d Name and identify the independent and dependent variables in your hypothesis.

4 Because of your model-building skills you have been asked to head a committee studying the job-training needs of a town’s labor force. The committee’s purpose is to identify ways to improve the quality of the town’s labor force to meet the needs of existing employers and to attract new employers to the area.

The committee’s first meeting is next week. A committee member who has no particular political power wants to survey citizens and community leaders; she has offered to prepare a survey for the meeting.

a Would you encourage her to bring a draft survey? Justify your decision. b Outline your agenda for the meeting.

5 Consider the problem of productivity in the United States. Identify recent publications that address the problem. Look at three of these publications. Find a definition for productivity.

Page 27 of 29Research Methods for Public Administrators

3/3/2021file:///C:/Users/Lamar%20Holmes/Desktop/VitalSource%20Bookshelf_%20Research%20Methods...

Identify the variables the authors use to study productivity. Create a tentative model to study productivity. Then state the purpose of your model. Sketch a schematic model that incorporates the variables you think are most important.

In class, work with three to five classmates. Compare your models, and jointly develop and sketch a model that you think could be tested.

6 Many communities must take water conservation measures in the summer. List strategies to get citizens to decrease their water usage. Develop a water conservation model that includes water conservation strategies and strategies for getting citizen compliance.

In class, work with three to five classmates. Compare your models, and jointly develop and sketch a model that you think should be tested.

7 Consider automobile accidents in the United States. Your instructor will divide the class into groups of three or four. Each member will locate and summarize a recent publication that studies the problem. Using this information, the group will complete the following:

a Identify three to five variables linked to automobile injuries or fatalities. b Create a preliminary model to arrive at a policy to reduce injuries and fatalities

resulting from automobile accidents. State the research question and a purpose for the model. Sketch a schematic model that incorporates the variables that you consider most important.

c State hypotheses to relate each independent variable in the model to the dependent variable.

d Prepare a presentation explaining and justifying your model.   Your instructor may provide alternative topics.

8 Select one of the following topics or a topic assigned by your instructor: Teenage pregnancy (incidence, policies, or programs) Juvenile crime (incidence, policies, or programs) Quality of drinking water Health insurance reform (policies, effectiveness) Municipal finance (innovations) Lobbying by nonprofit organizations (policies, activities)

a Use a web search engine to locate and explore a site relevant to the topic chosen. What information did you find at the site that could help you design or carry out a study?

b Using an electronic literature data base, search for research articles on the topic. How many entries did you receive? How many seem worth consulting?

c With a group of classmates, develop a guide to effective online searching.

Working With Data

Page 28 of 29Research Methods for Public Administrators

3/3/2021file:///C:/Users/Lamar%20Holmes/Desktop/VitalSource%20Bookshelf_%20Research%20Methods...

Three data files and three Word files accompany this text. The data files are: (1) an Excel spreadsheet, County Data File, containing several variables for all 100 counties in a state; data from a sample survey of county residents is provided in: (2) an Excel file, Bellco.xls, and (3) an SPSS data file, Bellco.sav. The data in the County Data File are from 2012 unless otherwise noted. The data were obtained from a central dataset of North Carolina data maintained by the state government. The sources of the data and the variables are described in a Word file, County Data File Notes, accompanying the data file. The items on the survey are included in a Word file: Belle County Survey. Another file, Focus Group Report, is a report of a focus group study. (Focus groups are discussed in Chapters 2 and 6.) While it is tempting to go immediately to a dataset and begin analysis, researchers should first state the questions they are trying to answer.

1 a Access the County Data File and variable descriptions and review the list of variables. State three questions a state or county administrator could answer by analyzing the data. These questions should address policies that would normally concern elected officials and administrators.

b Select two variables and state (1) one hypothesis with a positive relationship and (2) one with an inverse relationship.

b For each hypothesis identify a third variable from the dataset that could serve as a control variable.

2 Access the Belle County Survey Word file and review the questions and variables. State two research questions that would be of interest to someone who wanted to know about residents’ valuation of services.

a State two hypotheses: (1) one with a positive relationship and (2) one with a negative or inverse relationship. Ideally your hypotheses, when tested, should provide information to help answer the research questions.

Recommended for Further Reading Information technology is changing rapidly. Many texts exist to help Internet users with research, but most

become quickly outdated. Reference librarians are the best source for learning about new technologies and how to use them effectively. See Karen Hartman and Ernest Ackermann, Searching and Researching on the Internet and the World Wide Web, 5th ed. (Wilsonville, OR: Franklin, Beedle and Associates, 2010).

The following volume contains excellent discussions of model and theory building. The authors include exercises for students of different disciplines. See James Jaccard and Jacob Jacoby, Theory Construction and Model Building Skills: A Practical Guide for Social Scientists (New York, NY: The Guilford Press, 2010).

For information on writing a literature review, see: www.lib.ncsu.edu/tutorials/litreview; and J. Galvan, Guidelines for Writing Literature Reviews: A Guide for Students of the Behavioral Sciences, 4th ed. (Glendale, CA: Pyrczak Publishing, 2009). An Internet search will lead to additional guides. The NCSU cite has a good video narrated by an instructor. Galvan’s text has examples of literature reviews.

Page 29 of 29Research Methods for Public Administrators

3/3/2021file:///C:/Users/Lamar%20Holmes/Desktop/VitalSource%20Bookshelf_%20Research%20Methods...