Research Methodology & Policy Analysis IP2

Presentation

Understanding Research Methodology

In conducting social science research, the social scientist seeks to understand, and in turn explain, the world in which he or she lives. Rather than simply rely on what they observe and apply assumptions, beliefs, or general guesses to explain observations, social scientists approach this endeavor for an increased understanding using a systematic scientific method. Social scientists in the fields of homeland security, emergency management, and many others take this approach because it is their ultimate intention to go beyond their own personal understanding of why things happen. They want to inform others of these explanations and contribute to a greater body of knowledge. The purpose of developing, testing, and refining explanations for what is observed is to ultimately predict future behaviors or prescribe potential remedies for negative conduct in the form of policies.

Research methodology is comprised of the approaches, designs, plans, methods, and tools or instruments scientists will use to conduct their exploration. Remember that social science includes studying phenomena and activities related to emergency management, criminal justice, and homeland security. Consider an example to help understand this need for a systematic approach to studying your surroundings to devise a strategy or policy.

In this example, a planner known as Officer Lightly works in a local law enforcement department and is directed to develop a community policing plan with the intent to solicit and incorporate the assistance of citizens in reducing the annual number of property crimes each year. The former planner, Officer Grimly, had planned to develop a program based on his own beliefs about what would work. Officer Grimly simply briefed and published the plan to his department's leadership and then moved on to his next assignment. However, Officer Lightly is familiar with the scientific process and understands its value for tackling social science projects. Officer Lightly determines there is a wide assortment of objectives he might pursue, but he knows he needs to first start with a specific research question and then develop and test a hypothesis. Depending on the findings from his test of the hypothesis, he may proceed in his original direction or decide to take a different course.

Officer Lightly decides to craft two research questions and at least one hypothesis for each. He has formulated the following:

· Research Question 1 (R1): Where in the community do property crimes occur in the largest concentrations?

· Hypothesis 1 for R1: If an area in the community is low income, property crimes are higher.

· Research Question 2 (R2): What are citizens in areas of high crime currently doing in response to, or to protect against, property crimes?

· Hypothesis 1 for R2: If citizens act purposively to prevent property crime, they will not be victims of property crime.

Measuring Phenomena

In examining Officer Lightly’s variables, he recognizes that they may be vague and therefore hard to measure. For example, what constitutes low income or acting purposively? The researcher must operationalize these concepts. Operationalizing variables requires defining them and determining how they will be measured. The researcher may, for example, characterize a low-income area as one where 8 of 10 homes within a certain radius report annual family incomes below the federally defined poverty level of $19,000 for a family of four. There are many different techniques used to operationalize variables, and the previous example is only one of the potential ways for operationalization.

Presentation

Types of Research Design Approaches

In any study, providing an answer to the research question or idea is the primary objective. The research question focuses the investigation on a specific topic that guides the entire research design process, data collection, data analysis, and research evaluation. There are an innumerable number of possible research questions for any topic. Once the entire research process has been executed, new alternative questions may also become apparent.

Research design involves the researcher making decisions about how many samples to study, as well as to administer the dependent variable to eliminate or control threats to validity. The particular research design method to be used in a study and its application to the study must first be fully identified. The basic research design is fairly clear-cut and will most likely fit one of the following models:

· Correlational and causal-comparative research

· Survey research

· Qualitative and quantitative research

· Single-case research

Correlational and Causal-Comparative Research

Correlational and causal-comparative research are used to study phenomena involving the inherent characteristics of participants. However, there is an important difference between the two approaches: causal-comparative research focuses on making group comparisons, whereas the main focus of correlational research is to estimate the significance of the relationship between two variables. These differences in focus result in different types of conclusions that can be drawn (Mertens, 2005).

Survey Research

Survey research can be thought of as descriptive-research methods or as data collection methods used within other research designs (Mertens, 2005). Surveys can be used for a variety of reasons, and they are used especially in educational and psychological research. When developing an effective survey, a good first step is to write a brief purpose for the survey, in about 25 words or less. This purpose can then be expanded to include specific objectives of the survey. A survey researcher has a choice between several approaches: simple descriptive, cross-sectional, and longitudinal. The simple descriptive approach is a one-time survey administered over a short period of time. The cross-sectional approach describes the effects of groups over a specific period of time. The longitudinal approach considers one cohort or a number of groups over a long period of time (Mertens, 2005).

Qualitative and Quantitative Research

Qualitative research is designed to provide an in-depth description of a specific program, practice, or setting (Mertens, 2005). Quantitative research, on the other hand, is designed for the study of large groups using questionnaires, surveys, or observations.

Single-Case Research

Single-case research is particularly appealing to researchers and practitioners in education and psychology because it is based on the interest in determining the effectiveness of a single individual's intervention (Mertens, 2005).

Selecting Research Ideas

A strong research idea should be easy to justify. When choosing which study to pursue, it is important to ask the following questions:

· What potential impact will the results of the study have on the field?

· What will be gained from answering the research question?

· Who will benefit from the results of the study?

In life, there are times when someone will have an idea and think about that idea and how it could make a difference in the world. As a professional, a person might have to make an important decision but not have the tools or the knowledge to make the best choice. The development of these ideas and the need for statistical evidence are what fuel the research design process. As a researcher, a person can be assigned an idea and asked to work on it, or someone may have his or her own idea to pursue. Regardless of the circumstances, it is important to understand that research will always be in high demand and that all research projects must be undertaken with the highest levels of validity and reliability in mind.

Reference

Mertens, D. M. (2005). Research and evaluation in education and psychology: Integrating diversity with quantitative, qualitative, and mixed methods (2nd ed.). Thousand Oaks, CA: Sage.

Article

Article: Research Design

Research Design

Research design provides detailed plans that specify how data should be collected and analyzed. It covers two major areas: quasi-experimental and experimental research. Experimental design is a method of experimentation with the objective of implicitly or explicitly including the control of variables; researchers experiment by observing the effects of one or more variables on others under controlled conditions. In the quasi-experimental section, random assignments are not used to establish equivalent experimental and control groups (Trochim, 2006).

Research Objectives

Research design can be exploratory, descriptive, or experimental.

Exploration Research designs may have exploratory objectives. Exploratory design assumes that investigators have little or no knowledge about the research problem under study.

Description Descriptive design means depicting the characteristics of whatever is being observed. For investigation, important characteristics of study settings and persons are identified. Later, investigators may conduct experimental research where one or more of the described variables may be controlled and assessed. Descriptive designs are the most common in criminological research.

Experimentation Experimentation and experimental objectives are designed to demonstrate cause-effect relationships between variables. Because the relationship between variables takes a great deal of time and repeated research efforts to establish, investigators caution their readers to be careful of jumping into conclusions about cause-effect variable interrelationships. A key element with this design is experimental and control groups. A design is experimental if subjects are randomly assigned to treatment groups and to control (comparison) groups. Cook and Campbell (1979) mention 10 types of experimental design. Note that the control group may receive no treatment or it may be a group receiving a standard treatment (e.g., students receiving computer-supported classes versus those receiving conventional instruction). That is, the control group is not necessarily one to be labeled "no treatment."

Quantitative Research Quantitative research is the application of statistical procedures and techniques to data collected through surveys, including interviews and questionnaire administration. They are known for number crunches because a wide variety of sophisticated statistical techniques exist to describe what they have found. Thus, it involves the use of structured questions where the response options have been predetermined and a large number of respondents are involved. By definition, measurement must be objective, quantitative, and statistically valid. Simply put, it is about numbers—objective data (Champion, 2005). Thus, quantitative research involves investigation that heavily uses statistical procedures—statistical manipulations of data considered as primary in an effort to discover patterns of behavior. On the other hand, qualitative research is an investigation that does not rely heavily on statistical analyses—participant observation for the purpose of discovering underlying meanings and patterns of relationships.

Qualitative Research Qualitative research is the application of observational techniques or the analysis of documents as the primary means of learning about persons or groups and their characteristics. This type of research design is often called fieldwork, referring to the immersion of researchers into the lives and worlds of those studied. It involves collecting, analyzing, and interpreting data by observing what people do and say. Whereas quantitative research refers to counts and measures of things, qualitative research refers to the meanings, concepts, definitions, characteristics, metaphors, symbols, and descriptions of things (Champion, 2006).

Qualitative research is much more subjective than quantitative research and uses very different methods of collecting information, mainly through individuals, in-depth interviews, and focus groups. The nature of this type of research is exploratory and open-ended. Small numbers of people are interviewed in-depth and a relatively small number of focus groups are conducted. This type of research is often less costly than surveys. It is effective in acquiring information about people's communications, needs, and responses to specific communications (Hagan & King, 1992).

References  

Champion, D. J. (2005). Research methods for criminal justice and criminology (3rd ed.). Upper Saddle River, NJ: Prentice Hall.

Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design and analysis issues for field settings. Florence, KY: Cengage/Wadsworth.

Hagan, M., & King, R. P. (1992). Recidivism rate of youth completing an intensive treatment program in a juvenile correctional facility. International Journal of Offender Therapy and Comparative Criminology, 36, 349–358. 

Trochim, W. M. K. (2006, October). Research methods knowledge base. Retrieved March 30, 2009, from Web Center for Social Research Methods Web site: http://www.socialresearchmethods.net/kb/

Article

Operationalizing Variables

When examining variables, a researcher might decide that they may be vague and therefore hard to measure. The researcher must operationalize these concepts. Operationalizing variables, stated simply, is defining them and determining how they will be measured. The researcher may, for example, characterize a low income area as one where 8 of 10 homes within a certain radius report annual family incomes below the federally defined poverty level of $19,000 for a family of four. This is only one of a number of potential ways to operationalize this variable. Below are other examples of variable operationalization:

· Property crimes are higher.

· This is measured as follows: 4 in 10 properties (homes and businesses) in the given area have reported property crimes in the last 5 years.

· Act purposively to prevent property crime.

· This will be measured based on the types of activities people engage in with the intention to prevent their homes and businesses from being targeted. This will still seem vague because the researcher may not yet know what these activities will include.

· Property crimes are limited to incidents of vandalism to include graffiti. Incidents may include reported and nonreported vandalism.

· Victims of property crime is defined as reported cases of property crime.

Operationalizing a variable may require defining the parameters of each word within it. 

Article

Primary Approaches to Social Science Methods

The approaches or designs employed by social scientists to pursue their research fall primarily into two main categories: quantitative and qualitative. It makes sense that a research project might include elements of each of these; this is known as employing a mixed methods approach.  

Quantitative Methods Approach

Quantitative analysis, as the name implies, is concerned with collecting, measuring, and processing quantitative data. Quantitative analysis, or quant, is about numbers and logic using data that are fixed, objectively gathered, and examined. By starting with a problem or objective, the researcher brings together various facts and data, then applies what is already known and uses his or her own (or mathematical) logic to pursue answers or make recommendations regarding courses of action; this process is called convergent thinking. Think of this process as a kin to what a police detective does in attempting to solve a case as he starts with a problem then deductively pursues answers.

Quantitative analysis focuses on mathematical formulas, computer-based procedures, and statistical outputs. Although this output is interpreted by human beings, there are a number of rules that apply to guide the researcher in this interpretation. The data and information gathered is essentially known, but it is patterns in the data or observations that are looked for to determine predictors or to help explain phenomena. By combining the known information with rigorous mathematical procedures, the researcher can elicit findings to help inform his or her study and test his or her hypotheses. Examples of basic quantitative data might include income or temperatures.

The quantitative approach to scientific inquiry relies on a number of assumptions. One assumption is that social conditions have phenomena that are rooted in an objective reality. These facts then are able to be collected and considered in quantities. Another assumption is that variables can always be identified, and articulated and relationships between them can be measured quantifiably. An important assumption that quantitative analysis is tied to is the belief in the value of, and intent to capture, an outsider’s perspective, with the outsider being the objective scientist.

Quantitative analysis has several clear purposes. First, it is designed to produce findings that offer a generalized view about phenomena, relationships, patterns, or observations. Furthermore, the explanations or interpretations drawn from quantitative findings are intended to serve as general—therefore widely applicable—findings to support or refute a relevant theory or hypothesis. Another purpose of quantitative analysis is to yield findings that help predict additional analysis from the same data. In other words, when statistical analysis is performed following accepted protocols, it should produce a consistent output, which can also be reproduced by other social scientists, given the same data and same tested hypotheses. Ultimately, quantitative research focuses on providing causal explanations for variables within a relationship.

Qualitative Methods Approach

From the moment we are born, we are observing and processing information. In many cases, we predict an expected outcome based on what we have experienced firsthand in the past. We also observe what happens to others and discover lessons vicariously or listen carefully to what others say and heed advice. In these ways, we learn about our environment and then make sense of why things happen as they do. Students of social science may not see these methods for understanding our world as precise, yet they recognize that these help provide context and fundamental knowledge. 

Qualitative research relies on subjective methods to collect data or information that is also interpreted in subjective ways. This is not to say the interpretations or analyses are biased, rather the researcher focuses his or her examination on concepts, perceptions, and words, as communicated by participants. The data that is interesting to, and collected by, the qualitative researcher is dynamic and complex, potentially reflecting an ever-changing reality. The qualitative researcher employs divergent thinking in approaching a problem by formulating several or many innovative and unique possibilities that will serve as, or contribute to, solutions. The researcher’s thoughts diverge in several directions, drawing from multiple facets, to supply potential answers.

With a qualitative approach, reality is the result of social construction, with the premise being that we create our reality versus discovering it. The observations, collection of data, and interpretation of findings are all subjectively processed by the researcher, which naturally means the variables used in qualitative research are complicated and difficult to measure, and the perspectives are those of an insider.

The purpose for using qualitative methods includes providing a larger context for research and findings. Within the captured environment, the motivations and perspectives of participants or those observed are also considered in crafting a multidimensional profile. Qualitative research approaches complex social systems intending to engender understanding and meaning.

Mixed Methods Approach

It may seem obvious that these two primary approaches can be used together, but designing research that masterfully leverages both types of methods is a bit more complicated than it might first appear. The mixed methods approach blends quantitative and qualitative approaches to refine and complement each other by creating a new whole. There are many possibilities for selecting and integrating the methods, instruments, tools, and processes from each of the primary social research approaches, which will bring to a study the elements, data, and findings that are most useful for an exploration. The reason one might employ mixed methods is to tailor a complex research design to tackle a complex research problem.

Article

Types of Data

The type of data the researcher already has, seeks out or eventually uses can aid him in determining the best approach to employ. There are four types of data: nominal, ordinal, interval and ratio.

Nominal

Nominal data is, stated simply, named data. A nominal scale includes values such as one’s self-designation of their race. For example, a racial characterization cannot be ordered or measured, but by using names to collect data into categories, data can be identified and referenced.

Ordinal

Ordinal data refers to items that can be placed in an order. Income, for example, lists dollar figures for several participants. These income amounts can be placed in an order, say lowest to highest. High, medium and low may be ordered positions as well. An ordinal scale depicts a sequence. Another example might be reflected in questions on a survey that ask “How happy are you in your current position? One, Very happy; two, moderately happy; …” and so on. Ordinal data is not processed by arithmetic.    

Interval

Interval data is recognized as being equal distances apart. The common example is temperature as expressed in Fahrenheit degrees. Interval data cannot be multiplied or divided but it can be measured in quantities. It can also be represented on a scale.

Ratio

Ratio data can be multiplied and divided. On a ratio scale numbers might be multiples of other numbers. (4 is twice as much as 2.) Ratio data can be measured on a scale as well; for example the difference between 1 and 2 is the same as the difference between 3 and 4. Ratio data can be scale data. At the same it can be multiplied or divided and measured in quantities, so it is also quantitative data.

Article

Methods of Quantitative Analysis

Consider a community policing research project. The researcher has developed two research questions, has two hypotheses to test, and has also operationalized the variables. In the following example, just one variable is used to illustrate each of the major approaches. The variable used will be victims of property crime, which is limited to cases of vandalism (including graffiti).

A quantitative approach to testing this hypothesis might use criminal statistics, collecting those reports of vandalism/graffiti. The data may be statistically processed and analyzed, looking for demographic, geographic, the time of day, and other information. Patterns within any of these categories may help to inform the researcher about the parameters of her problem.

A qualitative approach to measuring this variable might include the officer looking for signs of vandalism and graffiti, and then informally interviewing residents in a vandalized neighborhood using open-ended questions, and potentially following up later. Her focus in her questions is to gather actual responses and capture the words of interviewees, but also to detect unspoken anxiety, anger, or detachment in residents.

A mixed methods approach might leverage both of these tactics as described above, or it might be used in a more integrated fashion. For instance, first the researcher looks at the crime stats. She then speaks to residents who made the reports in neighborhoods where reports are high. She then reexamines the stats to look for relationships to income levels, time of day, and various demographics. Again, she returns to the neighborhood and speaks with residents who have not made reports to ask them about their awareness regarding the vandalism and how its existence makes them feel in terms of security or trust in the police.

Why Choosing the Proper Approach Matters

Ultimately, the researcher should select and combine (if appropriate) the research methods that can best aid her in answering her research question, thereby achieving her objectives. To the extent possible, especially when the research will be the foundation for a policy or program, she should utilize a comprehensive methodology to ensure the resulting product’s value and applicability.

Activity

CHDS: Research Process and Methods – Developing a Research Plan

This lecture, provided by the Center of Homeland Defense and Security (CHDS), contains information on various issues and concepts with regard to the research design process.

Center for Homeland Defense and Security Research Process and Methods Lecture: Developing a Research Plan

Click on the following link to view Developing a Research Plan.

Click Here to View Lecture

Activity

CHDS: Research Process and Methods – Collecting Evidence: Introduction

This lecture, provided by the Center of Homeland Defense and Security (CHDS), contains information on various issues and concepts with regard to the research design process.

Center for Homeland Defense and Security Research Process and Methods Lecture: Collecting Evidence: Introduction

Click on the following link to view Collecting Evidence: Introduction.

Click Here to View Lecture

Activity

CHDS: Research Process and Methods – Conducting a Literature Review

This lecture, provided by the Center of Homeland Defense and Security (CHDS), contains information on various issues and concepts with regard to the research design process.

Center for Homeland Defense and Security Research Process and Methods Lecture: Conducting a Literature Review

Click on the following link to view Conducting a Literature Review.

Click Here to View Lecture

Resource Links

 Operationalization (http://www.experiment-resources.com/operationalization.html) Information on the operationalization of variables into measurable factors for experimental research.

 Department of Homeland Security (DHS) (http://www.dhs.gov) Official Web site of the Department of Homeland Security (DHS).

 Federal Emergency Management Agency (FEMA) (http://www.fema.gov/) Official Web site of the Federal Emergency Management Agency (FEMA). Contains information on types of disasters, planning, recovery, rebuilding, and federal assistance.

 Research Methods (http://allpsych.com/researchmethods/researchcontents.html) A virtual course on the fundamentals of research methods.

 Social Research Methods (http://www.socialresearchmethods.net/) A website for people involved in applied social research and evaluation. You'll find lots of resources and links to other locations on the Web that deal in applied social research methods. The site offers an online statistical advisor that will answer questions and lead you to an appropriate statistical test for your data. It also includes a resource guide for learning about structured conceptual mapping including links to general introductory materials, research and case studies illustrating the use of the method, and information about software. An online hypertext textbook on applied social research methods that covers everything you want to know about defining a research question, sampling, measurement, research design and data analysis is available, and an online workbook about manual (i.e., dice-rolling) and computer simulation exercises of common research designs, for students and researchers to learn how to do simple simulations.