Applying Basic Statistical Tests to a Research Scenario

MDWELLONS

4ea95e61ef9e1cf1a7840616eac28ac2.pdf

Home >Psychology homework help >Applying Basic Statistical Tests to a Research Scenario

Alignment and Misalignment Examples of Scenario Elements

In PSYC-8412 Research Foundations you build quantitative and qualitative research scenarios that include each of the

following key elements:

Social problem or phenomenon of interest

• Research problem

• Research purpose

• Research questions

• Theoretical or conceptual framework

• Research design

• Sampling strategy o Sampling criteria (qualitative only) o Data sources (qualitative only)

• Data collection method

• Variables (quantitative only)

• Analysis plan

• Trustworthiness (qualitative only)

All these key elements must logically align. Although the figure depicts

a linear flow, it is critical to understand that alignment is an iterative

process. For example, if after identifying a research problem and

research purpose additional research questions emerge, then the research

problem and purpose must be refined to align with the additional

research questions. Similarly, if variables of interest are identified that

are not represented in the research problem, purpose, or questions, and

that do not fit with the theoretical or conceptual framework, then those

elements will need to be refined to capture all of the variables of interest.

You will be piecing together your scenarios week-to-week, continually

adding new elements until a solid alignment of your research idea

emerges. Because of the iterative nature of alignment, you should not be

surprised that as a new element is added to your scenario that previous

elements may need to be modified to maintain alignment.

There are several ways for elements within a scenario to misalign, and it is not possible to provide examples of all

possible issues. In this document there are week-to-week example scenarios that demonstrate logically aligned elements

and some examples of the many ways elements become logically misaligned. Studying these will help you avoid some

common misalignment issues and understand how changing one element, sometimes even a single word, can affect

alignment. Below is one student’s reflection in Week 5 of the course:

I too struggled with the concept and terminology. For me, it is in fact the language that is used, and such

is definitely 'foreign' of sorts. As you go along though, it is all beginning to make sense. Initially the

feedback also was 'foreign' but now, going back and reviewing the question, answers, feedback are

beginning to all make sense. Even feedback that suggests that one simple word be changed makes sense

as what I submitted could possibly be misconstrued and cause the study to go in a different direction. I'm

beginning to understand how changing one simple word can make a difference. Research terminology

requires that things be concise and getting into the habit of relaying information properly makes all the

difference. I am beginning to speak 'research'. I believe it merely takes practice. One almost has to

develop a mental research template and think from another part of the brain and perspective. (K. Jackson,

Week 5 discussion post, September 29, 2018)

A table of contents is on the next page with active links to help you navigate the document.

Page 2 of 43

Table of Contents

WEEK 1 SCENARIO PIECES OF PRIMARY INTEREST ............................................................................................................................. 3

WEEK 1 QUANTITATIVE SCENARIO EXAMPLE ................................................................................................................................................... 4 Checking Week 1 Quantitative Alignment ........................................................................................................................................ 5

WEEK 1 QUALITATIVE SCENARIO EXAMPLE ..................................................................................................................................................... 6 Checking Week 1 Qualitative Alignment .......................................................................................................................................... 7

WEEK 2 SCENARIO PIECES OF PRIMARY INTEREST ............................................................................................................................. 8

WEEK 2 QUANTITATIVE SCENARIO EXAMPLE ................................................................................................................................................... 9 Checking Week 2 Quantitative Alignment ...................................................................................................................................... 10

WEEK 2 QUALITATIVE SCENARIO EXAMPLE ................................................................................................................................................... 11 Checking Week 2 Qualitative Alignment ........................................................................................................................................ 12

WEEK 3 SCENARIO PIECES OF PRIMARY INTEREST ........................................................................................................................... 13

WEEK 3 QUANTITATIVE SCENARIO EXAMPLE ................................................................................................................................................. 14 Checking Week 3 Quantitative Alignment ...................................................................................................................................... 15

WEEK 4 SCENARIO PIECES OF PRIMARY INTEREST ........................................................................................................................... 16

WEEK 4 QUALITATIVE SCENARIO EXAMPLE ................................................................................................................................................... 17 Checking Week 4 Qualitative Alignment ........................................................................................................................................ 18

WEEK 5 MIXED METHOD DESIGN .................................................................................................................................................... 19

MERGING QUANTITATIVE AND QUALITATIVE SCENARIOS INTO A MIXED METHODS DESIGN .................................................................................... 20

WEEK 6 SCENARIO PIECES OF PRIMARY INTEREST ........................................................................................................................... 21

WEEK 6 QUANTITATIVE SCENARIO EXAMPLE ................................................................................................................................................. 22 Checking Week 6 Quantitative Alignment ...................................................................................................................................... 23

WEEK 7 SCENARIO PIECES OF PRIMARY INTEREST ........................................................................................................................... 24

WEEK 7 QUANTITATIVE SCENARIO EXAMPLE ................................................................................................................................................. 25 Checking Week 7 Quantitative Alignment ...................................................................................................................................... 26

WEEK 8 SCENARIO PIECES OF PRIMARY INTEREST .......................................................................................................................... 28

WEEK 8 QUANTITATIVE SCENARIO EXAMPLE ................................................................................................................................................. 29 Checking Week 8 Quantitative Alignment ...................................................................................................................................... 30

WEEK 9 SCENARIO PIECES OF PRIMARY INTEREST ........................................................................................................................... 32

WEEK 9 QUALITATIVE SCENARIO EXAMPLE ................................................................................................................................................... 33 Checking Week 9 Qualitative Scenario Alignment ......................................................................................................................... 34

WEEK 10 SCENARIO PIECES OF PRIMARY INTEREST ......................................................................................................................... 36

WEEK 10 QUALITATIVE SCENARIO EXAMPLE ................................................................................................................................................. 37 Checking Week 10 Qualitative Scenario Alignment ....................................................................................................................... 38

WEEK 11 SCENARIO PIECES OF PRIMARY INTEREST ......................................................................................................................... 40

WEEK 11 QUALITATIVE SCENARIO EXAMPLE ................................................................................................................................................. 41 Checking Week 10 Qualitative Scenario Alignment ....................................................................................................................... 42

Page 3 of 43

Week 1

Scenario Pieces of Primary Interest

In Week 1 the focus is on identifying a social problem or phenomenon of interest and specifying a research problem,

research purpose, and research question for both a quantitative and qualitative research scenario. Typically, a thorough

review and understanding of the relevant literature is needed to identify a bona fide research problem. We know that you

have not already conducted such a thorough literature review. For purposes of this course, the research problem—

something the scholarly community does not know—simply needs to be plausible. On the following pages are an example

of a quantitative scenario and an example of a qualitative scenario with these four elements that demonstrate alignment

and misalignment.

Page 4 of 43

Week 1 Quantitative Scenario Example

Social Problem or Phenomenon of Interest: Critical thinking skills in the

United States have been reported to be lower than critical thinking skills in other

industrialized countries.

Research Problem: The scholarly community does not know the extent to

which critical thinking skills differ between graduating seniors from a high

school at which the curriculum was problem-based compared to a high school

with lecture-based curriculum.

Research Purpose: The purpose of this quantitative study is to determine the

extent of difference in critical thinking skill scores between graduating seniors

from high schools with a problem-based versus lecture-based curriculum.

Research Question: What is the extent of difference in critical thinking skill

scores between graduating seniors from high schools with a problem-based

versus lecture-based curriculum?

The social problem or phenomenon

can be, and typically is, broad.

The research problem narrows the

scope and identifies something

specific the scholarly community does

not know.

It conjectures that differences in

critical thinking skills may differ

based on type of curriculum.

It explicitly, or at least implicitly,

identifies the population of interest—

here it is graduating seniors.

The research purpose identifies the

study as quantitative and, otherwise,

simply repeats the research problem.

This could not be a qualitative study

because to determine differences in

critical thinking skills requires a

quantitative measure.

If the research purpose focused on

graduating seniors from public versus

private high schools, it would not be

aligned with the research problem,

which was about type of curriculum.

The research question simply replaces “the purpose of this

quantitative study is to determine” from the purpose with “what

is”—everything else is exactly the same.

If the research question asked about critical thinking scores and

overall GPA, it would not be aligned with the problem or purpose,

neither of which mention GPA.

Page 5 of 43

Checking Week 1 Quantitative Alignment

One way to visually check alignment is to create a table of the key concepts represented in each key element. If every

concept in one element is contained in the other elements, then it is aligned (see first table below). If, however, a concept

in one element is missing in the other elements, then it is misaligned (see second table below).

Aligned Concepts Across Elements

Research Problem Research Purpose Research Question

Critical thinking Critical thinking Critical thinking

Graduating seniors Graduating seniors Graduating seniors

Problem-based curriculum Problem-based curriculum Problem-based curriculum

Lecture-based curriculum Lecture-based curriculum Lecture-based curriculum

If the concepts of critical thinking, graduating seniors, problem-based curriculum and lecture-based curriculum

are included in the research problem; then, the research purpose and research question should also contain the same

concepts.

If these concepts appear in the

problem statement

The scholarly community does not

know the extent to which critical

thinking skills differ between

graduating seniors from a high

school at which the curriculum was

problem-based compared to a high

school with lecture-based curriculum.

Then, they should be contained in the

purpose statement

The purpose of this quantitative study

is to determine the extent of difference

in critical thinking skill scores

between graduating seniors from

high schools with a problem-based

versus lecture-based curriculum.

and in the research question

What is the extent of difference in

critical thinking skill scores between

graduating seniors from high schools

with a problem-based versus lecture-

based curriculum?

Misaligned Concepts Across Elements

Research Problem Research Purpose Research Question

Critical thinking Critical thinking Critical thinking

Graduating seniors Graduating seniors Graduating seniors

Problem-based curriculum Problem-based curriculum Problem-based curriculum

Lecture-based curriculum Lecture-based curriculum Lecture-based curriculum

Public high schools

Private high schools

GPA

Page 6 of 43

Week 1 Qualitative Scenario Example

Social Problem or Phenomenon of Interest: Critical thinking skills in the

United States have been reported to be lower than critical thinking skills in other

industrialized countries.

Research Problem: The scholarly community does not know what classroom

experiences contribute to students’ critical thinking skills or the developmental

range of experiences across elementary grade levels.

Research Purpose: The purpose of this qualitative study is to identify classroom

experiences that contribute to students’ critical thinking skills and to map the

developmental range of these experiences across elementary grade levels.

Research Question 1: What classroom experiences contribute to students’

critical thinking skills?

Research Question 2: What are the developmental range of experiences across

elementary grade levels?

The social problem or phenomenon

can be, and typically is, broad. And,

the same social problem or

phenomenon can be addressed by

either quantitative or qualitative

research.

It is the research problem that

determines a quantitative or

qualitative approach.

Previously, the quantitative scenario

focused on “differences” in critical

thinking skills between two groups.

Here, the research problem focuses on

“experiences.”

It conjectures that some classroom

experiences contribute to students’

critical thinking skills, and it

conjectures that the types of classroom

experiences may depend on the

developmental range of students.

It explicitly identifies the population

of interest as elementary grades.

The research purpose identifies the

study as qualitative to “identify”

classroom experiences and “map” the

developmental range.

If the research purpose focused on

students’ behavior or disciplinary

practices, it would not be aligned with

the research problem.

Because the research problem and research purpose are about two

different things—(a) classroom experiences, and (b) developmental

range of experiences—two separate research questions are needed. One

focused on the classroom experiences that contribute to students’

critical thinking skills, the other focused on the developmental range of

those experiences.

If a research question asked about experiences in accelerated classrooms

versus regular or remedial classrooms, it would not be aligned with the

problem or purpose, which only refer to elementary grade levels. If

understanding experiences in different types of classrooms is of interest,

then such would need to be incorporated in the research problem and

research purpose.

Page 7 of 43

Checking Week 1 Qualitative Alignment

The tables below extract the key concepts in the qualitative scenario and the additional concepts that were described that

would result in misalignment.

Aligned Concepts Across Elements

Research Problem Research Purpose Research Question

Classroom experiences Classroom experiences Classroom experiences

Developmental range of experiences Developmental range of experiences Developmental range of experiences

Critical thinking skills Critical thinking skills Critical thinking skills

Elementary grade levels Elementary grade levels Elementary grade levels

Misaligned Concepts Across Elements

Research Problem Research Purpose Research Question

Classroom experiences Classroom experiences Classroom experiences

Developmental range of experiences Developmental range of experiences Developmental range of experiences

Critical thinking skills Critical thinking skills Critical thinking skills

Elementary grade levels Elementary grade levels Elementary grade levels

Students’ behavior

Disciplinary practices

Accelerated classrooms

Regular classrooms

Remedial classrooms

Page 8 of 43

Week 2

Scenario Pieces of Primary Interest

In Week 2 the focus is on adding a theoretical or conceptual framework to the quantitative and qualitative scenarios

proposed in Week 1. Although not part of the scenarios, learning resources this week help you understand the primary

purpose of a literature review and to identify and evaluate scientific sources of information.

A theory or conceptual framework is necessary in designing a dissertation study and plays a key role in guiding the

research questions and interpreting the results of a study. On the following pages the quantitative scenario example and

the qualitative scenario example are carried forward from Week 1 with the theoretical or conceptual framework element

added to demonstrate alignment and misalignment.

Page 9 of 43

Week 2 Quantitative Scenario Example

(Highlighted element is new this week)

Social Problem: Critical thinking skills in the United States have been reported

to be lower than critical thinking skills in other industrialized countries.

Research Problem: The scholarly community does not know the extent to

which critical thinking skills differ between graduating seniors from a high

school at which the curriculum was problem-based compared to a high school

with lecture-based curriculum.

Research Purpose: The purpose of this quantitative study is to determine the

extent of difference in critical thinking skill scores between graduating seniors

from high schools with a problem-based versus lecture-based curriculum.

Research Question: What is the extent of difference in critical thinking skill

scores between graduating seniors from high schools with a problem-based

versus lecture-based curriculum?

Theoretical or Conceptual Framework: The cognitive and social constructivist

theory of learning is a dynamic and collaborative process in which students are

actively involved in their learning, rather than being passive listeners of a

lecture. This theory fits and guides the research question expecting differences in

critical thinking skills between students in a problem-based versus lecture-based

curriculum and will inform the interpretation of the results.

In this quantitative scenario

differences in critical thinking skills

are expected to be different depending

on type of curriculum: problem-based

or lecture-based.

A theoretical or conceptual framework

needs to be one that leads the

researcher to that expectation and that

will aid in the interpretation of the

results.

Here, the cognitive and social

constructivist theory of learning

provides the foundation for expecting

and interpreting differences in critical

thinking skills between those from a

school that uses a problem-based

curriculum versus lecture-based

curriculum.

A biopsychosocial framework would

not fit because there is nothing in the

research problem, purpose, or question

that refers to biological or

psychological factors that affect

critical thinking.

Nor would a gender roles conceptual

framework fit because there is nothing

prior that refers to gender roles

affecting critical thinking.

Page 10 of 43

Checking Week 2 Quantitative Alignment

The tables below extract the key concepts in the quantitative scenario and the additional concepts that were described that

would result in misalignment.

Aligned Concepts Across Elements

Research Problem Research Purpose Research Question

Critical thinking Critical thinking Critical thinking

Graduating seniors Graduating seniors Graduating seniors

Problem-based curriculum Problem-based curriculum Problem-based curriculum

Lecture-based curriculum Lecture-based curriculum Lecture-based curriculum

Theoretical or Conceptual Framework

Cognitive and social constructivist theory of learning guides the expectation and interpretation of differences in students’

critical thinking skills as a result of problem-based versus lecture-based curriculum exposure.

Misaligned Concepts Across Elements

Research Problem Research Purpose Research Question

Critical thinking Critical thinking Critical thinking

Graduating seniors Graduating seniors Graduating seniors

Problem-based curriculum Problem-based curriculum Problem-based curriculum

Lecture-based curriculum Lecture-based curriculum Lecture-based curriculum

Public high schools

Private high schools

GPA

Theoretical or Conceptual Framework

Biopsychosocial framework would not fit because there is nothing in the research problem, purpose, or question that

refers to biological or psychological factors that affect critical thinking.

Gender roles conceptual framework would not fit because there is nothing in prior elements that refers to gender roles

affecting critical thinking.

Page 11 of 43

Week 2 Qualitative Scenario Example

(Highlighted element is new this week)

Social Problem or Phenomenon of Interest: Critical thinking skills in the

United States have been reported to be lower than critical thinking skills in other

industrialized countries.

Research Problem: The scholarly community does not know what classroom

experiences contribute to students’ critical thinking skills or the developmental

range of experiences across elementary grade levels.

Research Purpose: The purpose of this qualitative study is to identify classroom

experiences that contribute to students’ critical thinking skills and to map the

developmental range of these experiences across elementary grade levels.

Research Question 1: What classroom experiences contribute to students’

critical thinking skills?

Research Question 2: What are the developmental range of experiences across

elementary grade levels?

Theoretical or Conceptual Framework: The cognitive and social constructivist

theory of learning, Kurfiss’s (1988) eight principles of critical thinking teaching

practices, and a developmental perspective form the conceptual framework for

this study. Each of these relate to identifying classroom experiences that

contribute to students’ critical thinking skills and the developmental perspective

also allows mapping of classroom experiences across elementary grade levels.

Each of these elements of the conceptual framework will also guide thematic

coding of lesson plans and classroom observations, and aid in the interpretation

of results.

In the research problem there is

conjecture that some classroom

experiences contribute to students’

critical thinking skills, and that the

types of classroom experiences may

depend on the developmental range of

students across elementary grades.

A theoretical or conceptual framework

needs to support the conjectures, guide

the specific research questions, and

serve as an aid in the interpretation of

the results.

Here, three elements make up the

conceptual framework.

1. The cognitive and social constructivist theory of learning

supports the conjecture that

critical thinking skills can be

enhanced by dynamic and

collaborative classroom

experiences that actively involve

students in the learning process.

2. Kurfiss’s (1988) eight principles of critical thinking teaching

practices.

3. Developmental perspective of classroom experiences.

The cognitive and social constructivist

theoretical lens, the eight principles of

critical thinking teaching practices,

and a developmental perspective will

guide data partitioning of lesson plans

and observations of classroom

exercises, and the coding and

interpretation of classroom experience

themes that contribute to critical

thinking skills.

Grounded theory would not fit the scenario because, first, grounded

theory is a research design not a theoretical or conceptual framework

and, second, the purpose of the study is to identify and

developmentally map classroom experiences that contribute to

students’ critical thinking skills, not to develop a theory, which is the

end goal of grounded theory.

The transformative emancipatory paradigm would not fit as a

conceptual framework because it is about giving voice to

marginalized groups, which is not a focus of the scenario’s research

problem, purpose, or questions.

Page 12 of 43

Checking Week 2 Qualitative Alignment

The tables below extract the key concepts in the qualitative scenario and the additional concepts that were described that

would result in misalignment.

Aligned Concepts Across Elements

Research problem Research purpose Research question

Classroom experiences Classroom experiences Classroom experiences

Developmental range of experiences Developmental range of experiences Developmental range of experiences

Critical thinking skills Critical thinking skills Critical thinking skills

Elementary grade levels Elementary grade levels Elementary grade levels

Theoretical or Conceptual Framework

Cognitive and social constructivist theory of learning supports the conjecture that critical thinking skills can be enhanced

by dynamic and collaborative classroom experiences that actively involve students in the learning process.

Kurfiss’s (1988) eight principles of critical thinking teaching practices dovetail with the cognitive and social

constructivist theory of learning to specifically identify teaching practices that actively involve students in the learning

process.

A developmental perspective takes into account the developmental range of teaching practices and actively involved

students across elementary grades.

Misaligned Concepts Across Elements

Research problem Research purpose Research question

Classroom experiences Classroom experiences Classroom experiences

Developmental range of experiences Developmental range of experiences Developmental range of experiences

Critical thinking skills Critical thinking skills Critical thinking skills

Elementary grade levels Elementary grade levels Elementary grade levels

Students’ behavior

Disciplinary practices

Accelerated classrooms

Regular classrooms

Remedial classrooms

Theoretical or Conceptual Framework

Grounded theory does not fit because it is a research design for the purpose of generating a new theory.

The transformative emancipatory paradigm does not fit because it is about giving voice to marginalized groups.

Page 13 of 43

Week 3

Scenario Pieces of Primary Interest

In Week 3 the focus is only on the quantitative scenario, adding a specific research design, sampling strategy (which

implies population), and data collection method that aligns with the research problem, purpose, and questions.

Campbell and Stanley (1963) described several specific quantitative designs that fall under the broad domains of pre-

experimental, quasi-experimental, experimental, and nonexperimental. Although you are not required in this course to

discuss threats to validity, identifying a specific design is critical because specific threats to validity are associated with

specific designs, not with the four broad domains or other generic terms such as exploratory, causal-comparative, cross-

sectional, or longitudinal.

The “Quantitative Research Design Concepts” pdf in the Week 3 Learning Resources area of the classroom distinguishes

between four commonly confused specific designs and addresses common quantitative sampling strategies and data

collection methods.

Page 14 of 43

Week 3 Quantitative Scenario Example

(Highlighted element is new this week)

Social Problem: Critical thinking skills in the United States have been reported to

be lower than critical thinking skills in other industrialized countries.

Research Problem: The scholarly community does not know the extent to which

critical thinking skills differ between graduating seniors from a high school at

which the curriculum was problem-based compared to a high school with lecture-

based curriculum.

Research Purpose: The purpose of this quantitative study is to determine the

extent of difference in critical thinking skill scores between graduating seniors

from high schools with a problem-based versus lecture-based curriculum.

Research Question: What is the extent of difference in critical thinking skill

scores between graduating seniors from high schools with a problem-based versus

lecture-based curriculum?

Theoretical or Conceptual Framework: The cognitive and social constructivist

theory of learning is a dynamic and collaborative process in which students are

actively involved in their learning, rather than being passive listeners of a lecture.

This theory fits and guides the research question expecting differences in critical

thinking skills between students in a problem-based versus lecture-based

curriculum and will inform the interpretation of the results.

Research Design: Comparing graduating seniors from a high school at which the

curriculum was problem-based compared to a high school with lecture-based

curriculum on critical thinking skills is a posttest only static group comparison

design because groups are compared on a current outcome after recently

experiencing different types of curriculum.

Sampling Strategy: A purposive sampling strategy will be used to invite only

graduating seniors (age 18 or over) from high schools with a problem-based or

lecture-based curriculum. Excluded will be freshmen, sophomores, juniors,

seniors under 18 years old, nongraduating seniors, and any high school senior in a

school with a curriculum other than problem-based or lecture-based.

Data Collection Method: Eligible students will be alerted to the opportunity to

participate through social media, which will contain a link to an online survey.

The research design must logically fit

the research problem, purpose, and

question. As previously noted these

three elements are aligned in focus on

comparing graduating seniors from

high schools with a problem-based

versus lecture-based curriculum on

critical thinking skills.

Because this is a current outcome after

recently experiencing different types

of curriculum, the appropriate design

is a posttest only static group

comparison.

It could not be an experimental

posttest only control/comparison

group design because students are not

randomly assigned to type of

curriculum.

It also could not be a quasi-

experimental nonequivalent

comparison group design because

there is no pretest.

The sampling strategy is purposive

because there are specific inclusion

criteria that also specifies the

population of interest.

It is not a convenience sample, even if

you conveniently invite students from

local high schools, because of the

specific inclusion criteria.

The data collection method of an

online survey fits the scenario because

an online survey can contain whatever

instrument is used to measure critical

thinking skills.

It could not be an unstructured

interview because reliability and

validity of a critical thinking skills

measure must ask each participant the

exact same questions.

Page 15 of 43

Checking Week 3 Quantitative Alignment

The tables below extract the key concepts in the quantitative scenario and the additional concepts that were described that

would result in misalignment.

Aligned Concepts Across Elements

Research Problem Research Purpose Research Question

Critical thinking Critical thinking Critical thinking

Graduating seniors Graduating seniors Graduating seniors

Problem-based curriculum Problem-based curriculum Problem-based curriculum

Lecture-based curriculum Lecture-based curriculum Lecture-based curriculum

Theoretical or Conceptual Framework

Cognitive and social constructivist theory of learning guides the expectation and interpretation of differences in students’

critical thinking skills as a result of problem-based versus lecture-based curriculum exposure.

Research Design

A pre-experimental posttest only static group comparison fits because a current outcome (critical thinking skill scores)

will be compared between two groups recently experiencing different types of curriculum.

Sampling Strategy

A purposive sampling strategy fits because there are specific eligibility criteria for participants.

Data Collection Method

The data collection method of an online survey fits the scenario because an online survey can contain whatever

instrument is used to measure critical thinking skills.

Misaligned Concepts Across Elements

Research Problem Research Purpose Research Question

Critical thinking Critical thinking Critical thinking

Graduating seniors Graduating seniors Graduating seniors

Problem-based curriculum Problem-based curriculum Problem-based curriculum

Lecture-based curriculum Lecture-based curriculum Lecture-based curriculum

Public high schools

Private high schools

GPA

Theoretical or Conceptual Framework

Biopsychosocial framework would not fit because there is nothing in the research problem, purpose, or question that

refers to biological or psychological factors that affect critical thinking.

Gender roles conceptual framework would not fit because there is nothing in prior elements that refers to gender roles

affecting critical thinking.

Research Design

An experimental posttest only comparison group design does not fit because students are not randomly assigned to type

of curriculum.

A quasi-experimental nonequivalent comparison group design does not fit because there is no pretest.

Sampling Strategy

A convenience sample does not fit because a convenience sample does not have specific participant eligibility criteria.

Data Collection Method

An unstructured interview does not fit because reliability and validity of a critical thinking skills measure must ask each

participant the exact same questions.

Page 16 of 43

Week 4

Scenario Pieces of Primary Interest

In Week 4 the focus is only on the qualitative scenario, adding a specific research design, sampling strategy (which

implies population), and data collection method that aligns with the research problem, purpose, and questions.

The “Qualitative Research Design Concepts” pdf in the Week 4 Learning Resources area of the classroom clearly

differentiates six common qualitative research designs, four common qualitative sampling strategies, and common

qualitative data collection methods.

For the qualitative design, do not use broad terms such as descriptive, exploratory, hermeneutic, or lived experience. Use

one of the six specific designs in the “Qualitative Research Design Concepts” pdf, and briefly describe how the design fits

your scenario. Also, be specific about the sampling strategy and data collection method with brief description of how each

fit your scenario.

Page 17 of 43

Week 4 Qualitative Scenario Example

(Highlighted element is new this week)

Social Problem or Phenomenon of Interest: Critical thinking skills in the

United States have been reported to be lower than critical thinking skills in other

industrialized countries.

Research Problem: The scholarly community does not know what classroom

experiences contribute to students’ critical thinking skills or the developmental

range of experiences across elementary grade levels.

Research Purpose: The purpose of this qualitative study is to identify classroom

experiences that contribute to students’ critical thinking skills and to map the

developmental range of these experiences across elementary grade levels.

Research Question 1: What classroom experiences contribute to students’

critical thinking skills?

Research Question 2: What are the developmental range of experiences across

elementary grade levels?

Theoretical or Conceptual Framework: The cognitive and social constructivist

theory of learning, Kurfiss’s (1988) eight principles of critical thinking teaching

practices, and a developmental perspective form the conceptual framework for

this study. Each of these relate to identifying classroom experiences that

contribute to students’ critical thinking skills and the developmental perspective

also allows mapping of classroom experiences across elementary grade levels.

Each of these elements of the conceptual framework will also guide thematic

coding of lesson plans and classroom observations, and aid in the interpretation of

results.

Research Design: A case study design will be used to focus on the bounded

system of classroom experiences across elementary grade levels. A case study is

appropriate because multiple sources of information (classroom observations,

review of lesson plans) will be used for case-based themes common across and

unique to elementary grade levels.

Sampling Strategy: The purpose is not to identify rare classroom experiences

that contribute to students’ critical thinking skills or all possible experiences, but

the normal and usual experience, so a typical sample will be sought.

Data Collection Method: Data will be collected by observation of activities in

the natural classroom setting and from archival document review of teachers’

lesson plans.

The sampling strategy is “typical”

because normal and usual classroom

experiences are sought.

It is not a “unique” sampling strategy

because interest is not in rare

classroom experiences that contribute

to students’ critical thinking skills,

though rare events could emerge from

the analysis.

The research design must logically fit

the research problem, purpose, and

question. As previously noted these

three elements are aligned in focus on

identifying and mapping classroom

experiences that contribute to

students’ critical thinking skills across

elementary grade levels.

A case study design fits this focus,

particularly because multiple sources

of information will be used to

understand the bounded system of the

classroom.

It could not be a narrative design

because it is not about using people’s

stories to understand the classroom

experiences, but about direct

observation and document review of

lesson plans.

To identify and map the

developmental range of classroom

experiences, direct observation of

classrooms and document review of

lesson plans fit the scenario.

A structured survey asking teachers

about the frequency of use of

Kurfiss’s (1988) eight principles of

critical thinking teaching practices

could provide valuable information,

but such would make the scenario a

quantitative study.

Page 18 of 43

Checking Week 4 Qualitative Alignment

The tables below extract the key concepts in the qualitative scenario and the additional concepts that were described that

would result in misalignment.

Aligned Concepts Across Elements

Research problem Research purpose Research question

Classroom experiences Classroom experiences Classroom experiences

Developmental range of experiences Developmental range of experiences Developmental range of experiences

Critical thinking skills Critical thinking skills Critical thinking skills

Elementary grade levels Elementary grade levels Elementary grade levels

Theoretical or Conceptual Framework

Cognitive and social constructivist theory of learning supports the conjecture that critical thinking skills can be enhanced

by dynamic and collaborative classroom experiences that actively involve students in the learning process.

Kurfiss’s (1988) eight principles of critical thinking teaching practices dovetail with the cognitive and social

constructivist theory of learning to specifically identify teaching practices that actively involve students in the learning

process.

A developmental perspective takes into account the developmental range of teaching practices and actively involved

students across elementary grades.

Research Design

A case study design fits the scenario particularly because multiple sources of data 9classroom observations, document

review of lesson plans) will be used to understand the bounded system of the classroom.

Sampling Strategy

The sampling strategy is typical because normal and usual experiences are sought.

Data Collection Method

To understand what classroom experiences contribute to students’ critical thinking skills direct observations of

classrooms and document review of lesson plans will be conducted.

Misaligned Concepts Across Elements

Research problem Research purpose Research question

Classroom experiences Classroom experiences Classroom experiences

Developmental range of experiences Developmental range of experiences Developmental range of experiences

Critical thinking skills Critical thinking skills Critical thinking skills

Elementary grade levels Elementary grade levels Elementary grade levels

Students’ behavior

Disciplinary practices

Accelerated classrooms

Regular classrooms

Remedial classrooms

Theoretical or Conceptual Framework

Grounded theory does not fit because it is a research design for the purpose of generating a new theory.

The transformative emancipatory paradigm does not fit because it is about giving voice to marginalized groups.

Research Design

A narrative design does not fit because the scenario is not about using people’s stories to understand classroom

experiences.

Sampling Strategy

A unique sampling strategy does not fit because interest is not in rare classroom experiences.

Data Collection Method

A structured interview of teachers’ frequency of use of Kurfiss’s (1988) eight principles of critical thinking teaching

practices would make it a quantitative study.

Page 19 of 43

Week 5

Mixed Method Design

From a common social problem or phenomenon of interest you have developed a quantitative research scenario and a

qualitative research scenario. Neither of these were created specific to a particular mixed method design. In Week 5 the

focus is on discussing how your quantitative and qualitative scenarios might be combined for a mixed methods study.

Creswell and Creswell (2018) describe in Chapter 10 three basic types of mixed methods designs: convergent, explanatory

sequential, and exploratory sequential. Pay particular attention to pages 217–218, 221–222, and 224. Creswell and

Creswell also discuss factors important in choosing a design (pp. 236–239) including a recommended design based on the

study’s objectives and expected outcomes (see Table 10.3, p. 237, particularly the rows related to the three basic types).

It is possible, perhaps likely, that your quantitative and qualitative scenarios do not fit any of the three basic types of

mixed methods design because they were not created with mixed methods in mind. The example quantitative and

qualitative scenarios described in this document do not fit a mixed method. First, the populations of interest are different,

graduating high school seniors in the quantitative scenario and elementary grade levels in the qualitative. Second, the

quantitative is about measuring and comparing critical thinking skill scores, while the qualitative is about identifying and

mapping classroom experiences that contribute to critical thinking skills.

Even though the separate scenarios do not fit as mixed methods, it’s possible to add something to one or the other scenario

to align with one of the three basic types of mixed methods design. The most critical addition, from which all other

elements would logically follow, is a second research problem. Examples are provided in the table on the next page. In

your Week 5 discussion post it is sufficient to simply add an additional qualitative research problem to your quantitative

research problem, or an additional quantitative research problem to your qualitative research problem, identify the

appropriate mixed method design, and briefly explain how the design fits with the original and added research problem.

The three basic mixed method designs are described below:

Explanatory Quantitative-Qualitative Sequential Design: In this design the plan for the qualitative portion depends on

and must be conducted after analysis of the quantitative results. The purpose of the qualitative portion is to explain

(hence, explanatory) “confusing, contradictory, or unusual” (Creswell 7 Creswell, 2018, p. 222) quantitative findings, or

to gain a better understanding of selected participants’ thought process, perspectives, or experiences related to the subject

matter of the quantitative portion.

Exploratory Qualitative-Quantitative Sequential Design In this design the plan for the quantitative portion depends on

and must be conducted after analysis of the qualitative results. For example, the quotes, codes, and themes from the

qualitative analysis might reveal new variables or constructs of interest from which items and scales could be developed

and tested.

Convergent Design: In this design the quantitative and qualitative data are typically collected at the same time and

address the same concepts or constructs, one for the purpose of quantitatively relating or testing group differences on the

concepts or constructs, the other to gain a rich, in-depth understanding of the concepts or constructs from participants’

perspectives.

Page 20 of 43

Merging Quantitative and Qualitative Scenarios Into a Mixed Methods Design

Original Research Problem Additional Research Problem Mixed Method Design

Quantitative: The scholarly

community does not know the extent

to which critical thinking skills differ

between graduating seniors from a

high school at which the curriculum

was problem-based compared to a

high school with lecture-based

curriculum.

Qualitative: The scholarly

community does not know the

classroom experiences of graduating

high school seniors in problem-based

versus lecture-based curricula that

might explain differences, or not, in

critical thinking skills.

Explanatory quantitative-

qualitative sequential: The

qualitative piece follows the

quantitative piece for the specific

purpose of trying to explain critical

thinking skill score differences based

on classroom experiences.

Qualitative: The scholarly

community does not know what

classroom experiences contribute to

students critical thinking skills or the

developmental range of experiences

across elementary grad levels.

Quantitative: The scholarly

community does not know the extent

to which public and private

elementary schools differ in the use of

classroom experiences and the

developmental range of experiences

that contribute to critical thinking

skills.

Exploratory qualitative-

quantitative sequential: After

identifying in the qualitative piece the

classroom experiences and

developmental range of experiences,

the coded list is used to create a

quantitative study to explore the

frequency of use of these in public

and private elementary schools.

Quantitative: The scholarly

community does not know the extent

to which critical thinking skills differ

between graduating seniors from a

high school at which the curriculum

was problem-based compared to a

high school with lecture-based

curriculum.

Qualitative: The scholarly

community does not know the types

of challenges high school teachers

face in creating lesson plans and

classroom activities that might

contribute to students’ critical

thinking skills.

Convergent: The qualitative data

does not depend at all on the

quantitative results. The quantitative

and qualitative data are simply

merged to provide a fuller

understanding of the critical thinking

skills social problem

Page 21 of 43

Week 6

Scenario Pieces of Primary Interest

In Week 6 the focus is only on the quantitative scenario, adding a description of each variable, each variable’s range of

measured values, the level of measurement of each variable, and the identification of each variable as independent or

dependent that aligns with the research problem, purpose, and questions.

For example, sex has two levels (male, female) and according to Stevens’s (1946) framework is a nominal level of

measurement. If you asked males and females how many minutes of news programming they watched or listened to, on

average, each day, the number of minutes would be a ratio level of measurement. If you expected males and females to

differ in the number of minutes, then sex would be the independent variable and number of minutes the dependent

variable.

Make sure you familiarize yourself with Stevens’s four levels of measurement and the other Week 6 Learning Resources.

Page 22 of 43

Week 6 Quantitative Scenario Example