Quantitative Research Process Matrix

Omoba

Week7and8Quantitativeandmixedmethodsdesigns.pptx

Home >Business & Finance homework help >Operations Management homework help >Quantitative Research Process Matrix

Dr. Tina Abrefa-Gyan

Res 7000

Research Methods

Indiana Tech Global Leadership Program

12/05/2021

Agenda

Any Announcements

Assignment Overview

Quantitative Research

Questions/Comments

Sue L. T. McGregor, Understanding and Evaluating Research. © SAGE Publications, 2018.

Quantitative Research Methodology

Research question is concerned with finding causal connections (or non-causal associations) between variables related to the phenomenon--so, need statistical and numerical data

To describe, predict, control, or explain

Sue L. T. McGregor, Understanding and Evaluating Research. © SAGE Publications, 2018.

Methodological Reminder

Mixed Methods Research Methodology

Assumes the research question cannot be answered unless there is both qualitative and quantitative data-- so, need both participants’ words and statistical numbers

To confirm, augment, or contradict each other (and more)

Sue L. T. McGregor, Understanding and Evaluating Research. © SAGE Publications, 2018.

Difference Between Predetermined and Emergent Research Designs

Quantitative

Predetermined, fixed research design plan is based mostly on reconstructed logic:

Qualitative

A broader and less restrictive concept of research design, in which researchers use “logic-in-use” as well as ‘reconstructed logic’ to accommodate the “design in use” principle:

Sue L. T. McGregor, Understanding and Evaluating Research. © SAGE Publications, 2018.

Difference Between Predetermined and Emergent Research Designs

Quantitative

This logic of research is based on organizing, standardizing, and codifying research into explicit rules, formal procedures, and techniques so others can follow the same linear plan, reconstruct the study, and get the same results

Qualitative

This is called an emergent research design wherein the original plan changes as the research unfolds, meaning it is a nonlinear plan

Sue L. T. McGregor, Understanding and Evaluating Research. © SAGE Publications, 2018.

Main Differences Between

Qualitative And Quantitative Inquiries

Sue L. T. McGregor, Understanding and Evaluating Research. © SAGE Publications, 2018.

Qualitative Inquiry

Assumes subjective reality is socially constructed and subjective

Research is value-bound and researcher’s values are accounted for

Researcher is the primary instrument (observations, interviews)

Quantitative Inquiry

Assumes there is an objective reality ready to be discovered

Research is value-neutral and the researcher’s values are muted

Uses inanimate instruments (e.g., scales, questionnaires, checklists, tests)

Sue L. T. McGregor, Understanding and Evaluating Research. © SAGE Publications, 2018.

Qualitative Inquiry

Contextualizes findings and applies ideas across contexts

Portrays natural settings and contexts

Thematic, patterned analysis of data

Power in rich descriptions and detail

Quantitative Inquiry

Generalizes results from a sample to a population

Manipulates and controls variables

Statistical analysis of data

Statistical power

Sue L. T. McGregor, Understanding and Evaluating Research. © SAGE Publications, 2018.

Qualitative Inquiry

Understands perspectives (empathetic) and exploration

Widely, deeply examines phenomenon

Focus on quality, essence, nature

Uses inductive then deductive logic

Quantitative Inquiry

Provides causal explanations and predictions

Narrowly tests specific hypotheses

Focuses on quantity and how much

Uses deductive then inductive logic

Sue L. T. McGregor, Understanding and Evaluating Research. © SAGE Publications, 2018.

Qualitative Inquiry

Searches for patterns and looks for complexity

Uses purposive sampling

Research design is emergent, evolving

Data are words, images, categories

Nonlinear, iterative, and creative analysis

Quantitative Inquiry

Analyzes discrete components looking for the norm

Uses random sampling

Research design is predetermined

Data are numbers (minor use of words)

Linear, standardized, and prescribed analysis

Sue L. T. McGregor, Understanding and Evaluating Research. © SAGE Publications, 2018.

Qualitative Inquiry

Some studies create theory from the findings

Generates understandings from patterns

Faces conceptual complexity

Strives for trustworthy, credible data

Quantitative Inquiry

Use theory to ground the study and interpret results

Test hypotheses that are born from theory

Faces statistical complexity

Strives for reliable and valid data

Sue L. T. McGregor, Understanding and Evaluating Research. © SAGE Publications, 2018.

Quantitative Methods

Sue L. T. McGregor, Understanding and Evaluating Research. © SAGE Publications, 2018.

Two Overarching Quantitative Research Designs

Experimental

Relatively small and randomly chosen sample, which is measured before and after a controlled treatment, striving to establish “cause and effect” relationships between variables

Non-experimental

Usually a larger sample, where subjects are measured (observed) once (sometimes over time) in their natural setting, striving to establish natural “associations” between natural variations

Sue L. T. McGregor, Understanding and Evaluating Research. © SAGE Publications, 2018.

Main Differences Between (Non)Experimental Designs

Experimental

Manipulate variables

Randomly assign to comparison groups (control and treatment)

Or randomly assign a treatment to existing groups

Nonexperimental

No variable manipulation; rely on naturally occurring variations

Use existing groups

Sue L. T. McGregor, Understanding and Evaluating Research. © SAGE Publications, 2018.

Main Differences Between (Non)Experimental Designs

Experimental

Or find two existing groups, one experiencing the condition and one not (quasi)

Seek cause and effect relationships

Nonexperimental

Seek natural associations (within existing groups)

Seek natural characteristics within existing groups (descriptive)

Sue L. T. McGregor, Understanding and Evaluating Research. © SAGE Publications, 2018.

Types of Quantitative Research Designs

Predetermined Research Design

Researcher adheres to a formal plan with no deviation

Types of Quantitative Research Designs

Descriptive (nonexperimental)

Correlational

Comparative

Sue L. T. McGregor, Understanding and Evaluating Research. © SAGE Publications, 2018.

Types of Quantitative Research Designs

Experimental

Quasi-experimental

Predictive exploratory

Survey (nonexperimental)

Sue L. T. McGregor, Understanding and Evaluating Research. © SAGE Publications, 2018.

Ensuring Validity and Reliability in a Quantitative Research Design

Validity--need to ensure that the study measured what it intended to measure

Reliability--need to ensure that the predetermined research plan for the study (steps for sampling, data collection, and analysis) can be repeated by someone else and get the same results

Sue L. T. McGregor, Understanding and Evaluating Research. © SAGE Publications, 2018.

Ensuring Validity and Reliability in a Quantitative Research Design

Generalizability--need to ensure that any conclusions can be extrapolated to those not included in the study but are from the same population (because the study sampled the population)

Sample means using a small amount or part of something as an example of the character, features, or quality of the whole

Sue L. T. McGregor, Understanding and Evaluating Research. © SAGE Publications, 2018.

Components of Quantitative Research Designs

Instruments, apparatus, procedures

Sampling

Ethical considerations

Data collection

Sue L. T. McGregor, Understanding and Evaluating Research. © SAGE Publications, 2018.

Components of Quantitative Research Designs

Data analysis

Account for validity, reliability, and generalizability

Data security and management

Limitations of predetermined design

Sue L. T. McGregor, Understanding and Evaluating Research. © SAGE Publications, 2018.

Instruments, Procedures, Or Apparatus Used To Obtain Data From The Sample

Survey instruments

Questionnaires

Tests

Experiments

Intervention protocols

Sue L. T. McGregor, Understanding and Evaluating Research. © SAGE Publications, 2018.

Instruments, Procedures, Or Apparatus Used To Obtain Data From The Sample

Pre-developed and validated assessment tools

Procedures, with sufficient details so that readers can evaluate the relevance of and/or duplicate them

Any materials or special devices (mechanical or electronic) designed for this study

Examples include software, drugs, machines, mechanical models, or visual stimuli

Sue L. T. McGregor, Understanding and Evaluating Research. © SAGE Publications, 2018.

Define Variables

Theoretical/conceptual definitions

Authors need to define variables as understood by a particular theory or a conceptual framework used in the study

Operational definition

Authors need to identify the steps taken to manipulate a specific variable in their study

Sue L. T. McGregor, Understanding and Evaluating Research. © SAGE Publications, 2018.

Define Variables

Saying the room was made really dark is too vague

If people want to replicate the study, they need to know exactly how dark, and how the illumination was changed (e.g., dimming the lights, closing the blinds)

Sue L. T. McGregor, Understanding and Evaluating Research. © SAGE Publications, 2018.

Quantitative Sampling

Authors need to describe the subjects or sources from where data are collected:

Number of people, artifacts, or animals

Criteria for selecting or enlisting subjects or artifacts--sampling techniques (see next slide)

When they were selected

Sue L. T. McGregor, Understanding and Evaluating Research. © SAGE Publications, 2018.

Quantitative Sampling

Authors need to describe the subjects or sources from where data are collected:

How subjects or treatments were assigned to groups and/or treatments (if applicable)

Demographics of the sample, if necessary

Why people or sources were excluded from the sample frame

Sue L. T. McGregor, Understanding and Evaluating Research. © SAGE Publications, 2018.

Quantitative Sampling

Authors need to describe the subjects or sources from where data are collected:

Sample attrition and any impact on study

Statistical power calculations (sample size and effect--need large enough sample to meet acceptable level of chance)

Sue L. T. McGregor, Understanding and Evaluating Research. © SAGE Publications, 2018.

Probability Sampling

Random sampling means equal chances for each item or subject to be selected (chance mitigates researcher's bias)

Systematic sampling entails taking every Kth case from the entire list of the population

Stratified sampling involves identifying several strata (subgroups) and randomly drawing from each one

Cluster sampling involves breaking the population into clusters or groups that are outwardly similar (they are from same population) but are inwardly different

Sue L. T. McGregor, Understanding and Evaluating Research. © SAGE Publications, 2018.

Nonprobability Sampling

Convenience sampling means subjects are selected because of their convenient accessibility and proximity to the researcher

Sue L. T. McGregor, Understanding and Evaluating Research. © SAGE Publications, 2018.

Quantitative Ethical Considerations

Honesty--do not fabricate, falsify, suppress, or misrepresent data. Do not deceive participants (withhold or mislead) unless justified to answer the research question

Data integrity--alert potential users of the limits of the data’s reliability and applicability; researchers should not exaggerate the accuracy or statistical explanatory power of their data

Objectivity--researchers have to interpret and present results objectively (bias free)

Sue L. T. McGregor, Understanding and Evaluating Research. © SAGE Publications, 2018.

Quantitative Ethical Considerations

Promote fairness and truthfulness when applying the scientific method

Informed consent, respect for human dignity--tell participants about the study (purpose, procedures, risks, and benefits), and give them the opportunity to voluntarily participate or decline (avoid coercion); ensure they can withdraw with no penalty

Sue L. T. McGregor, Understanding and Evaluating Research. © SAGE Publications, 2018.

Quantitative Ethical Considerations

Humane consideration of participants (civilized, compassionate, moral, and intellectual concern)

Protect participants from undue intrusion, distress, indignity, physical discomfort, personal embarrassment, or other forms of harm and risk

Confidentiality, anonymity, privacy--protect identity of participants in the study (unless they give permission otherwise)

Properly train all research personnel (especially for clinical trials and experiments)

Sue L. T. McGregor, Understanding and Evaluating Research. © SAGE Publications, 2018.

Quantitative Data Collection

Quantitative data are numbers. Once the sample is secured, researchers have to collect data (numbers) from the sample frame, using any instruments, procedures, and apparatus designed or procured for their study. Authors have to report all of this in their paper.

Experiments

Clinical trials

Computer simulations

Sue L. T. McGregor, Understanding and Evaluating Research. © SAGE Publications, 2018.

Quantitative Data Collection

Observing, counting and recording well-defined events

Obtaining data from information management systems

Surveys with closed-ended and/or open-ended questions (paper-based, face-to-face, telephone, web-based, wireless devices)

Content analyses

Sue L. T. McGregor, Understanding and Evaluating Research. © SAGE Publications, 2018.

Quantitative Data Collection

Primary Data

Collected by the researcher in order to address their research question

Secondary Data

Collected previously by someone else, not for this particular research question, but are used by the researcher, who manipulates preexisting statistical data using computational techniques

For example, census data, national longitudinal survey data, and panel data like the Labour Force Survey

Sue L. T. McGregor, Understanding and Evaluating Research. © SAGE Publications, 2018.

Authors Need to Report

When the quantitative data were collected, where, and by whom (themselves, co-researchers, trained assistants)

Indicate if their instrument was pilot tested before its final implementation

A pilot test is a small scale trial that helps identify issues with instructions, unclear items, formatting, typographical errors and/or other issues

Sue L. T. McGregor, Understanding and Evaluating Research. © SAGE Publications, 2018.

Quantitative Data Analysis

Because quantitative data are numbers, they are analyzed using descriptive statistics, inferential statistics, or both (see chapter 12)

Descriptive statistics summarize just the current data set and inferential statistics (causation and association) aim to draw conclusions about subjects or artifacts outside of the current data set

Sue L. T. McGregor, Understanding and Evaluating Research. © SAGE Publications, 2018.

Quantitative Data Analysis

Authors must identify which statistical procedures they used to analyze their numerical data

For example, central tendencies, variance, relative standing, regression analyses, ANOVAs, MANOVAs, structural equation modelling, hierarchical linear modelling, multiple discriminant analysis, multidimensional scaling, or some combination

Sue L. T. McGregor, Understanding and Evaluating Research. © SAGE Publications, 2018.

Quantitative Data Analysis

Authors must:

Provide a solid rationale for their statistical design choices, and cite supportive literature for their data analysis decisions

Identify which statistical software program was used (e.g., SPSS, SAS, Tableau, LISREL, Minitab, Excel)

Explain how they tried to ensure the statistical significance of their data (descriptive or inferential statistical conventions)

Quantitative Data Analysis

Authors must:

Explain how they handled missing data, and why its absence did not undermine the validity of their analysis (see chapter 13)

Explain how they cleaned their data set, which involves detecting and correcting (or removing) corrupt, incomplete, irrelevant, or inaccurate parts of the data set

Chapter 13 discusses reporting quantitative results

Quantitative Research Design Limitations Affect the Reach of the Conclusions

Internal Validity

Study design mistakes

Measurement and instrument errors

Inadequate statistical power (e.g., sample was too small)

External Validity

Sampling size and bias

Characteristics of the sample that compromise generalizing the results

Group selection or assignment bias

The Hawthorne effect

Treatment effects

Experimental effects

Temper the Limitations

Authors of quantitative papers should choose one or two salient limitations with the greatest threat to validity and focus on them by:

Identifying and explaining why these limitations are not a huge issue (i.e., they do not negatively affect the conclusions drawn in the discussion)

Provide reasons why the research design flaw could not be overcome

Temper the Limitations

Do not use an apologetic tone; rather, use a humble and a conservative tone

Mixed Methods

Philosophical Reminder

A mixed method study draws on both qualitative and quantitative research methodologies (assumptions), and then chooses respective methods to sample, collect, and analyze data from each strand (and then, in varying degrees, integrates the two strands for richer insights)

Philosophical Reminder

The term mixed methods applies only to studies that employ both quantitative and qualitative methods, and only to instances when qualitative and quantitative methods are used within a single research project or study

The assumption is that the research question(s) cannot be answered unless both types of data, their analyses, and their interpretation are available

Data Integration

It is one thing to conduct a study using methods from both strands. It is another thing to weave together insights gained from analyzing each strand’s data. Richer conclusions can be drawn if authors strive for data integration, possible in varying degrees

Integrate is Latin integrare, “making whole”

Integration involves mixing things that were previously separate

Data Integration Principle

Quantitative Stand + Qualitative Strand = Integrated Data

Authors must explain how they addressed this principle in their mixed methods research design plan (to what degree):

Integrated during data collection

Integrated during data analysis or interpretation

Integrated across the whole study

No integration at all

Approaches to Integrating the Two Strands

Separately, but at the same time:

Concurrent: with integrated inferences

Parallel: with separate inferences

Embedded: one form supports the other

Transformative: use theoretical lens in combination with mixing methods

Approaches to Integrating the Two Strands

Sequential: one after the other

Fully mixed or integrated: interactive mixing of approaches throughout the whole study

Conversion: change one form to another

Integration Anticipates Two Types of Inference

Strand-specific inferences

Draw inferences from the data generated in each separate strand

Meta-inferences across strands

Integrating initial strand-specific inferences into inferences that apply across the entire data set

Meta means encompassing all

Inference means drawing conclusions from evidence or reasoning

Data Integration Example

When statistically analyzing survey results, researchers noticed that subjects indicated they often failed to report credit card debt when claiming bankruptcy. From this result, the researchers inferred that the subjects were breaking the law. When reading the interview transcripts from selected subjects, a theme emerged that the subjects did not know they had to report credit card debt when claiming bankruptcy. In fact, they were very unknowledgeable about the legality of the entire process. The researchers thus inferred that they were irresponsible consumers for not availing themselves of this information.

Data Integration Example

However, upon reading a White Paper report to Senate, researchers learned that bankruptcy trustees often offered truncated and short explanations of the bankruptcy process, if any at all. Trustees justified this approach by citing a hole in the legislation. When taking all data sources into account, and examining their initial reactions to the data, the researchers revised their initial inferences and reached a meta-inference instead, concluding that the bankruptcy system and law were flawed, rather than consumers being irresponsible. They would not have come to this meta-insight (across all data sets) if they had just relied on the survey, the interview data, or the document analysis.

Prioritizing the Two Strands

Authors should indicate if they prioritized either the qualitative or quantitative strand in their research design, relative to how to best answer their research question

They may want to use Morse’s (1991) notation system when reporting their mixed methods design. UPPERCASE letters are used to indicate prominence, and lowercase letters indicate less dominant or lower priority methods for the particular research question. A plus (+) sign means the methods occurred at the same time, and an arrow (⇨) indicates sequence

Prioritizing the Two Strands

Example

QUAL+ QUAN means both are equally important, and occurred at the same time

QUAL ⇨ Quan: qualitative was privileged, followed with quantitative

QUAN ⇨ Qual: quantitative was privileged, followed with qualitative

Organizing the Methods Section

Authors need to choose an approach that makes sense of the data, and convinces readers that the research plan for sampling, data collection, and analysis is meaningful, structured, and coherent

Four possible organizational strategies:

Chronological (the order that things occurred)

Organizing the Methods Section

A most-to-least important structure within the chronological approach

General-to-specific

Major components of the research design, identified with subheadings (e.g., sampling, data collection, data analysis)

Technical Issues When Describing Mixed Methods

Use past tense

Third person for quantitative, and first and second person for qualitative

Ehen writing about the quantitative strand, authors are encouraged to use passive voice because it places the focus on what was done, and not who did it

Qualitative sections should use active voice because it describes who did something, not what was done

Concluding Thought … A Methodological Reminder

Quantitative Research Methodology

Research question is concerned with finding causal connections (or non-causal associations) between variables related to the phenomenon--so, need statistical and numerical data

To describe, predict, control, or explain

Concluding Thought … A Methodological Reminder

Mixed Methods Research Methodology

Assumes the research question cannot be answered unless there is both qualitative and quantitative data--so, need both participants’ words and statistical numbers

To confirm, augment, or contradict each other (and more)