lecture notes thinking in grade one leraners.pdf

Chapter I: Introduction

Background

On June 3, 2020, the New Jersey Board of Education adopted new social studies

standards that were to inform education for all public-school students in grades kindergarten

through grade twelve. The social studies standards were created to prepare students because:

Today's challenges are complex, have global implications, and are connected to people,

places, and events of the past. The study of social studies focuses on a deep

understanding of concepts that enable students to think critically and systematically about

local, regional, national, and global issues. The authentic learning experiences that enable

students to apply content knowledge, develop social studies skills, and collaborate with

students from around the world prepare New Jersey students for college, careers, and

civic life (New Jersey Department of Education, 2020).

This purposeful shift meant that the New Jersey Department of Education recognized that

the world was becoming more interconnected and that students in New Jersey needed to prepare

for a more interconnected and interdependent future. Against this background, this study arises.

The researcher's decision to employ Hess' Cognitive Rigor Matrix for evaluating higher-order

thinking in K–8 social studies standards, with the assistance of artificial intelligence (AI), was

influenced by her background as a science educator who implemented the next-generation

science standards (NGSS). Through her tenure teaching science, she witnessed firsthand the

transformative effect of the NGSS in enhancing educational rigor and fostering critical thinking

abilities among students. This experience prompted her to question whether the social studies

standards could achieve a comparable level of cognitive complexity. By examining the cognitive

demands inherent in the social studies standards, this study aimed to uncover the extent to which

these standards equipped students to navigate the intricacies of a globally interconnected world,

in line with the objectives outlined by the New Jersey Department of Education.

The education of children has traditionally been a national enterprise, designed

primarily to socialize and prepare individuals for participation in local communities and in

autonomous nation-states (Suárez-Orozco & Qin-Hilliard, 2004). However, the world is now

radically interconnected and challenged by social, economic, and environmental problems that

spill across political and geographic divides (Hull & Hellmich, 2018). Fostering international

mindedness has long been a goal of specialized international schools (Hill, 2012). If public

school students are to remain competitive, then public education must also shift to a more

international-minded approach to teaching. This requires educators to give students the ability to

think globally, which means they must think critically in order to understand their role in an

interconnected world.

The 2020 intent, spirit, and mission for the New Jersey social studies standards align with

the United Nations' sustainable development goals (SDGs), specifically targeting Goal 4:

Education (Goal 4). The SDGs emphasize the importance of inclusive and quality education as a

means to empower individuals and promote social justice. Additionally, the key to implementing

sustainable development goals is a commitment to ensuring systems thinking is embedded in all

levels of education (Stafford-Smith et al., 2016). This shows a commitment to giving students

agency and the tools they need to navigate the world effectively. By helping students understand

the complex interconnections between concepts, the state of New Jersey's Department of

Education is promoting a more holistic and nuanced approach to problem-solving and critical

thinking.

As schools continue to educate children and prepare them for an uncertain future,

educators and students are confronted with the reality of an interconnected global society. A

society that will require students to see themselves beyond their local community and one that

requires them to understand how decisions made locally have an impact thousands of miles away

and across geo-political lines. This context calls for a profound transformation in education.

Educators are tasked with nurturing global, critical thinking, and social consciousness in the

hearts and minds of learners. Paulo Freire, a pioneering advocate for social justice in education,

saw education as the foundation of politics. He promoted education as a foundational tool to

guide political decisions. To him, critical thinking should be used to make pedagogy relevant,

meaningful, and empowering.

In fact, his legacy at the present moment is prophetic and invaluable for navigating a dark

time in history, a time that points to a mix of hope and despair. The intersecting crises of

economics, health, climate change, politics, racism, and democracy appear apocalyptic,

especially with the rise of far-right and updated versions of fascist politics emerging across the

globe (Giroux, 2021). In the midst of all these big problems, the ideas from individuals such as

Paulo Freire show how important education is in creating a fair and just world. By giving

students the skills to think critically and be aware of the world around them, schools are not just

teaching them how to deal with the world's complexity; they are preparing them to be the ones

who'll make the world a better place.

Statement of the Problem

The world is evolving and changing quickly. Three-quarters of American companies say

they have difficulty recruiting the right people, with critical thinking among the top

requirements, according to the Society for Human Resource Management (Charlton, 2019).

Entering the world of global competition, the emphasis is on the need to prepare students to be

communicative, collaborative, creative, innovative, to think critically and analytically, and to be

able to effectively solve real-world problems. With higher-order thinking skills, which are

essential for absorbing knowledge as well as for work performance, students will become

effective communicators, critical and dynamic thinkers, competent problem solvers, and career

experts (Živkoviüa, 2016). This is important as schools have to prepare students for uncertain

futures. Technological displacement of labor is predicted to be significant in the future; if left

unchecked, it will lead to catastrophic societal unemployment levels (Bruun & Duka,

2018). Additionally, while it is imperative to prepare students to be successful in a changing

world professionally, it is as imperative to provide them with the tools to create a just world

where all people can attain success and freedom. This requires educators to encourage students

to think critically (Hackman, 2005).

Intent and Spirit

In New Jersey, students receive social studies instruction from Kindergarten through

Grade 12. Today's challenges are complex, have global implications, and are connected to

people, places, and events of the past. The study of social studies focuses on a deep

understanding of concepts that enable students to think critically and systematically about local,

regional, national, and global issues. The intent of the standards is to create authentic learning

experiences that enable students to apply content knowledge, develop social studies skills, and

collaborate with students from around the world while preparing New Jersey students for

college, careers, and civic life. Additionally, it should be noted that the natural integration of

technology in social studies education allows students to overcome geographic borders, apply

scientific and mathematical analysis to historical questions and contemporary issues, appreciate

cultural diversity, and experience events through the examination of primary sources. The 2020

New Jersey student learning standards in social studies are also informed by national and state

standards and other documents such as the college, career, and civic life (C3) framework for

social studies state standards. The National Center for History Education, National Council for

Social Studies, National Council for Geographic Education, Center for Civic Education, National

Council on Economic Education, National Assessment of Educational Progress, and the

Partnership for 21st Century Skills also influenced their creation (New Jersey Department of

Education, 2020).

Mission

Social studies education provides learners with the knowledge, skills, attitudes, and

perspectives needed to become active, informed, and contributing members of local, state,

national, and global communities (New Jersey Department of Education, 2020). If educators and

school administrators are responsible for nurturing students to adopt a more global perspective,

it's crucial to evaluate whether the standards that have been developed align with the desired

goals.

Purpose of the Study

The primary purpose of this research study was to assess the alignment of the 2020 New

Jersey student learning standards for social studies with Hess' Cognitive Rigor Matrix by using

artificial intelligence (AI) techniques. Although this research incorporated the use of AI, it is

important to note that the AI results were reviewed and validated by the researcher. The use of

AI allowed the researcher to determine if the standards promoted the expectation of high-order

thinking for students. Specifically, this study looked to:

● Utilize AI-powered natural language tools to analyze the language used in the standards,

identifying, and categorizing specific keywords, phrases, or concepts that align with the

criteria defined in Hess' Cognitive Rigor Matrix.

● Create AI prompts, which were specific instructions or inputs provided to an AI model to

generate a response or perform a task. In this study, AI was tasked with categorizing

content as lower-order or higher-order thinking based on linguistic patterns and context

and Webb's depth of knowledge levels.

Research Questions

The study was guided by the overarching research question, which sought to investigate

whether the 2020 New Jersey student learning standards for social studies in Grades K–8 aligned

with the cognitive rigor matrix proposed by Hess. Do these standards effectively provide

students with the necessary framework for developing the competencies required for global

citizenship, specifically by examining the presence, complexity, and frequency of higher-order

thinking opportunities within these standards? This study examined how the language in these

standards matched up with Hess' definition of higher-order thinking. The study also assessed

how cognitively complex the standards are and how many chances they give for higher-order

thinking using Hess' Cognitive Rigor Matrix. The following additional questions served as a

guide for this research:

Research Question 1: In what way(s) does the language found on the 2020 New Jersey

student learning standards for social studies in Grades K–8 compare to Hess' Cognitive Rigor

Matrix definition of higher-order thinking?

Research Question 2: How cognitively complex are the 2020 New Jersey student learning

standards for social studies in Grades K–8 as defined by Hess' Cognitive Rigor Matrix?

Research Question 3: How frequently do the 2020 New Jersey Student learning standards

for social studies in Grades K–8 include higher-order thinking opportunities as identified in Hess'

Cognitive Rigor Matrix?

Significance of the Study

This research study holds significant implications for the field of education and

curriculum development. It investigates whether the New Jersey social studies standards for

grades K–-8 foster critical thinking and global thinking in order to better prepare students to

understand and thrive in the interconnected world they live in. It also looks to explore how

advanced AI technologies can enhance the evaluation of educational standards in terms of

cognitive rigor, efficiency, and objectivity. By leveraging AI, this study aimed to provide

valuable insights into the alignment of the standards with higher-order thinking skills, ultimately

enhancing the rigor and effectiveness of social studies education. It also served to assess how AI

could provide efficiency and scalability when analyzing large volumes of educational materials.

AI results were then reviewed and validated by the researcher using the Framework Method and

the Mayring Step Model.

Theoretical Framework

The theoretical framework of this study incorporated four prongs that guided the

researcher, as evidenced in Figure 1. The researcher looked at critical thinking, global thinking,

and pedagogy as interconnected concepts that play a vital role in education. Critical thinking and

global thinking are essential competencies in the 21st-century education landscape. This is

because critical thinkers are able to deal with ambiguity, question biases, and apply complex

thought processes when they encounter a challenge. They can think of creative improvements to

systems and organizations and see solutions in the context of the world around them (Weeks,

2023). In order to better understand and assess critical thinking skills, the researcher was guided

by Bloom's taxonomy, Hess' Cognitive Matrix, and Webb's depth of knowledge. Global thinking

was informed by research on sustainable futures, interconnectedness, and social justice. The

work of Paulo Freire's Pedagogy of the Oppressed played a pivotal role in guiding this

research. Pedagogical understanding was informed by the work of John Dewey's How We Think.

Additional key aspects of the study were thinking as well as cognitive complexity. The works of

John Dewey and Paulo Freire, in particular, offered invaluable theoretical perspectives that

closely intersect with the exploration of learning, cognition, and pedagogy as they relate to

learning.

The theoretical framework of the study also included the integration of AI, text analysis,

natural language processing, and automated coding techniques to assess the cognitive rigor of the

New Jersey social studies standards in Grades K–8. As it relates to AI, feature extraction refers

to the process of identifying and isolating important pieces of data from a larger dataset. This

involved using AI to evaluate student learning outcomes. These processes were then validated by

the researcher, who applied the Framework Method and the Mayring Step Model to review the

findings.

Figure 1

Theoretical Framework

Note. Theoretical Framework. Own Work.

Definition of key Terms

Artificial intelligence (AI) is defined as one of the technological innovations that

happened to replace the manual work that is done by humans in various fields. Artificial

intelligence is a branch of science and technology that creates intelligent machines and computer

programs to perform various tasks that require human intelligence (Aghion et al., 2019).

Artificial intelligence refers to the general ability of computers to emulate human

thought and perform tasks in real-world environments, while machine learning refers to the

technologies and algorithms that enable systems to identify patterns, make decisions, and

improve themselves through experience and data (Artificial Intelligence (AI) vs. Machine

Learning, 2023).

Automated coding refers to AI using algorithms to automatically learn insights and

recognize patterns from data (Artificial Intelligence (AI) vs. Machine Learning, 2023)

Bloom's taxonomy addresses three domains of learning (the cognitive, psychomotor, and

affective) the lower-order learning of knowledge, comprehension, and application and the

higher-order learning of analysis, synthesis, and evaluation (Nentl & Zietlow, 2008).

Cognitive complexity refers to the state or quality of a thought process that involves

numerous constructs with many interrelationships among them. Such processing is often

experienced as difficult or effortful (American Psychological Association, 2010).

Critical thinking refers to reasonable, reflective thinking that is focused on deciding what

to believe (Ennis, 1985).

Global thinking refers to thinking routines that foster understanding and appreciation of

today's complex globalized world (Mansilla, 2016).

Hess' Cognitive Rigor Matrix refers to a tool that has significant potential to enhance

instructional and assessment planning and practices at the classroom level (Hess et al., 2009).

The CRM superimposes two different cognitive complexity measures—Bloom's taxonomy and

Webb's depth of knowledge—to produce a means of analyzing the emphasis placed on curricular

materials, instructional focus, and classroom assessment (Hess et al., 2009).

Higher-order thinking refers to a situation in which for the individual or group

concerned, one or more appropriate solutions have yet to be developed (Research Advisory

Committee of the National Council of Teachers of Mathematics, 1988).

Natural language processing (NLP) refers to the branch of computer science—and more

specifically, the branch of artificial intelligence or AI—concerned with giving computers the

ability to understand text and spoken words in much the same way human beings can. It

combines computational linguistics; rule-based modeling of human language, with statistical,

machine learning, and deep learning models (International Business Machines, 2024)

Webb's depth of knowledge refers to how deeply students must know, understand, and be

aware of what they are learning in order to attain and explain answers, outcomes, results, and

solutions. It also designates how extensively students are expected to transfer and use what they

have learned in different academic and real-world contexts (Francis, 2017).

Assumptions

This study made five assumptions that were carefully considered. The study assumed that

education played a vital role in shaping society and that learning standards, as set forth by the

New Jersey Department of Education, could assist educators and school administrators in

creating learning environments where students could see themselves as part of an interconnected

world. This study also assumed that critical thinking, as delineated by Hess' Cognitive Rigor

Matrix, could be effectively employed to evaluate the manifestation of global

thinking. Additionally, the study assumed that the work of Paulo Freire on social justice in

education offers important insights for helping students understand their role in a complex and

interconnected world. John Dewey's pedagogical principles were also assumed to contribute to a

deeper understanding of how children learn.

Limitations

The assessment of global thinking through AI and NLP techniques has not been

extensively used in educational frameworks and may have limitations in capturing nuanced

aspects of cognitive development. Additionally, Hess' Cognitive Rigor Matrix is used to measure

cognitive complexity. It's important to note that while this matrix was created in 2009, it has not

been as extensively studied as some other frameworks for assessing cognitive complexity and

critical thinking.

Delimitations

There were three delimitations that were considered in the context of this paper. The first

is that the research concentrated on evaluating the New Jersey student learning standards for

social studies, specifically targeting cognitive rigor alignment. The study also did not employ a

comprehensive review of all potential pedagogical approaches to teaching global

thinking. Lastly, AI and NLP were used to assess cognitive complexity and linguistic patterns

using a computer-aided coding technique.

Organization of the Study

A major theme of this dissertation was the issue of cognitive complexity within the New

Jersey learning standards for Grades K–8. In addition, it introduced AI as a coding tool, which is

in line with AI's integration across several fields. The literature review in Chapter 2 encompasses

the study's theoretical framework, prior research on higher-order thinking, and four main

research components. Chapter 3 examines the research design and methodology in detail while

offering a detailed analysis of Artificial Intelligence's involvement in the coding process and the

reliability mechanisms used to ensure it. Results, including the data discovered during the study,

are presented in Chapter 4. Chapter 5 summarizes the findings, discusses their implications, and

suggests future research along with potential policy recommendations.

Chapter 2: Literature Review

Technological advancements are taking place at an alarming pace in the 21st

century. These may increase exponentially as time moves forward. This rapid pace of

technological progress and globalization creates a constantly shifting socio-political landscape,

which future generations will be left to navigate. To be successful, children must be able to

adapt, comprehend, and negotiate unprecedented complexities. This extraordinary change

leading to unfettered globalization, which was supposed to bring the world together and provide

greater prosperity, inevitably seems to be leading to economic inequality and globalization

(Watson, 2017). Watson (2017) shared how globalization has led to an increase in international

trade, investments, and technological advancements. These have created significant economic

growth and even development in many parts of the world. He also made a point to discuss how

this wealth has not been evenly distributed among the masses or throughout the world. Although

globalization was once described as a great equalizer and an opportunity to develop previously

undeveloped countries, the wealth has not been evenly distributed, what it has actually done, is to

create wider economic disparities both between and within countries. Watson explained that

globalization has benefited from access to highly skilled workers around the world, but low-

skilled workers have faced job losses and stagnant wages due to competition from cheaper labor

markets abroad. This has led to an increase in income inequality not just in developing countries

but even in many developed countries. But income inequality isn’t the only concern. Other

concern associated with globalization include:

• Cultural Homogenization: Tomlinson described cultural homogenization as a trend in

which different cultures begin to look like each other due to the process of globalization.

Instead of varied, rich cultures, there is less cultural diversity. This leads to the loss of

local traditions. Customs lose their distinctiveness. Global media, products, and ideas are

being spread across wider geographies, reaching various cultures that then look to the

media-depicted ideals and begin to minimize the importance of cultural diversity. While

he acknowledged how global connection could lead to the creation of an understanding,

he also warned of the danger associated with the decline of diversity (Tomlinson, 2009).

• Exploitation of Labor: In his work, Silver (2013) explained how exploitation of labor

was bad for a number of reasons. Specifically, he highlighted how it undermined human

rights, how it brought inequality to society. Silver went on to talk about how it widened

the gap between the rich and poor. This gap decreases consumer purchasing power, and it

undermines the dignity of work and the workers (Silver, 2013).

• Environmental Degradation: Shiva (2000) spoke on the implications of deforestation,

pollution, and industrialization, which cause deteriorating ecosystems and loss of

biodiversity, as well as intensifying climate change. According to Shiva, environmental

degradation does not only hurt the environment; it endangers human health and culture.

Shiva emphasized the need for sustainable practices and how the world must turn to eco-

friendly policies to avoid more damage and to assure the well-being of our planet and its

inhabitants.

• Loss of Sovereignty: Held and McGrew (2009) spoke of the dangers that threaten the

autonomy of individual countries through mechanisms like international corporations,

trade agreements, and global governing bodies. These groups have so much power,

money, and influence that they can actually challenge how democracies work. What's

more, they don't always have to follow the same rules or face the same consequences as

you, as individuals. Consider how, in the United States of America, corporations are

deemed people and can contribute large sums of money to elections, yet if a company is

responsible for a death or the health of individuals, there are not any individuals who are

criminally charged.

These threats create a sense of urgency to develop an education model that does not rely

on the old agrarian model of banking information. It requires a model of learning that could

empower students, that will give them greater agency. To do so, one must create learning

environments that promote both critical and global thinking. A model of learning is needed that

may give students agency over the content and one that can equip them to face future challenges.

Considering Hess’ and Bloom's perspectives on higher-order thinking, curricula must incorporate

tasks that require analysis, evaluation, and invention in place of rote learning and memorization.

This fosters intellectual growth and prepares students for real-world problem-solving, thereby

nurturing citizens capable of ethical reasoning and constructive debate in a complex, globalized

world.

Conscientização, critical consciousness, is a concept popularized by Brazilian educator

and philosopher Paulo Freire (Campus Suite, n.d.). It refers to the process of gaining a

comprehensive understanding of the world, which enables the perception and exposure of social

and political contradictions. Through conscientização, individuals are not only aware of systemic

and structural injustices, but they are also motivated to act against them. For Freire, reflection

without subsequent action is meaningless. People need to reflect upon their learning in order to

better understand their circumstances and act to change them. This enhanced comprehension

through reflection and critical thinking ensures that subsequent actions are well-informed and

effective. As a result of Freire's pedagogy, oppressed people were empowered through reflection

and practice. In Freire's view, to reach a higher level of understanding and application, it was

necessary to synthesize several viewpoints and perspectives in order to reach invention and

innovation. He believed that critical thinking enabled individuals to go beyond accepting

information at face value by equipping them with the means to challenge dominant narratives,

question authority, and seek alternatives. It was his belief that empowered individuals were in a

better position to take decisive and significant action (Ko, 2024).

This mixed-methods study aimed to assess global thinking in the 2020 New Jersey

student learning standards for social studies in Grades K–8. The choice to work with these grades

was purposeful. In kindergarten through fifth grade, students develop their foundational skills for

learning.

By middle school, students have a broader understanding of the world, and they possess

more advanced cognitive abilities that allow them to build on the skills they have established in

their primary years. Assessing global thinking is becoming increasingly important. Today's

students will live in a more interconnected global society and a society whose technological

advancements and progress are changing at alarming rates. As a result of this, students need to

develop a deeper understanding of diverse cultures and global issues. These issues include the

development of sustainable economics and navigating multiple perspectives. The literature

review spanned several key areas, including higher-order thinking and cognitive complexity,

using Hess' Cognitive Rigor Matrix, which provided a framework for evaluating the cognitive

demands of learning tasks. The research examined global thinking, drawing on a wealth of

previous studies and theoretical frameworks. The researcher embarked on a thorough literature

review. The aim was to sift through empirical studies and theories that shed light on higher-order

thinking and how global thinking is measured. This took place while exploring various

theoretical frameworks connected to these concepts. The research questions shaped the literature

review. During the literature review, the definitions of higher-order thinking as they appear in

both educational curricula and scholarly discussions were addressed. This was also done while

addressing the concept of global thinking and how it is related to higher-order thinking and

cognitive complexity, as the mission and vision of the New Jersey Department of Education's

social studies standards in Grades K–12 speak to them. Particularly, the review focused on the

application of higher-order thinking within the 2020 New Jersey student learning standards for

social studies for Grades K–8.

Literature Search Strategy

The purpose of this chapter is to examine and review previous research and literature.

Boote and Beile's (2005) guidelines were the basis for the literature research procedures.

According to Boote and Beile (2005), a comprehensive, sophisticated literature review is both

the starting point and the source of ideas for significant, fruitful research. In addition, according

to Boote and Beile (2005), for education research to be useful and meaningful, it must be

cumulative; it must build on and learn from prior research and scholarship on the topic. To

advance this study, the researcher set out to not only read related previous research but also

review and analyze the literature. The researcher's literature review included online searches

from the Seton Hall University Database, SAGE, ProQuest Databases, EBSCO, Google Scholar,

New Jersey Department of Education websites, previous dissertations, and peer-reviewed

articles. The search terms included higher-order thinking, cognitive complexity, pedagogical

rigor, global thinking, global awareness, social studies standards, Webb's depth of knowledge,

Bloom's taxonomy, Hess' matrix, social justice, and artificial intelligence. The criteria for

inclusion of literature included, (a) peer-reviewed original research, (b) dissertations, (c)

government reports, (d) peer and non-peer-reviewed literature global thinking, (e) theoretical

literature, (f) seminal works, (g) studies published within the last 50 years.

Overview of Current Literature

The research in the area of how learning standards affect global thinking in students and

the extent to which cognitive complexity is embedded in learning standards was limited. The

first part of the literature review centered on critical thinking. Much of the research expressed the

need to develop complex analytical thinking within students and emphasized a need to develop

21st century skills. 21st century skills include the 4 Cs: critical thinking, creativity, collaboration

and communication. A focus was placed on better understanding Bloom's taxonomy, Hess'

Cognitive Rigor Matrix, and Webb's depth of knowledge. For this reason, the literature review

focused on analyzing theories to provide an understanding of higher-order thinking and cognitive

complexity using Hess' Cognitive Rigor Matrix.

Global thinking was the subject of the second part of the literature review. Due to a lack

of literature in the area, the researcher reviewed literature on global thinking and focused on

sustainable futures, interconnectedness, and social justice. Paulo Freire's work Pedagogy of The

Oppressed was a pivotal part of this research. The third purpose was to review the history of

pedagogy, most importantly as explained and seen by John Dewey, and how it relates to child

development, thinking, and cognition. The researcher also devoted time to better understanding

artificial intelligence. Feature extraction was a crucial component of this research because one of

the main functions of AI is to extract information and skills from a large dataset. In this case, it

may be the New Jersey student learning standards for social studies for Grades K-8. Finally, this

review concludes by analyzing the New Jersey student learning standards for social studies in

grades K-8, their influence on curriculum, and the demand for global student thinking.

As children move from kindergarten through high school graduation, the content and

organization of their education have a significant impact on their capacity to contribute to a

world that is thriving and healthy (Dernbach, 2009). For this reason, it is critical to recognize that

the knowledge and skills children acquire throughout their educational journey shapes their

perception of the world in which they live and their potential to address urgent societal

challenges.

In order to cultivate new knowledge and foster innovative ways of thinking among young

people and adults, critical thinking and global citizenship are critical components. Giving

students agency to achieve economic prosperity, actively participating in democratic processes,

promoting justice and equity, and concurrently regenerating the health of ecosystems is the

fundamental basis of all life and production (Cloud, 2005). In global citizenship, learners learn

how social, economic, and environmental dimensions are interconnected and develop the tools to

make informed decisions about complex global issues.

Education in system dynamics and systems thinking are crucial in this situation. Systems

thinking is a comprehensive approach to viewing the world around us as parts of a whole

(Arnold & Wade, 2015). This thinking allows for the analysis of the interactions between the

parts of the system to better understand those interactions to live successfully within the system

(Norqvist & Ärlestig, 2021). These instructional frameworks direct students to investigate both

the relationships and interdependencies among discrete system components and the discrete

system as a whole. Global thinking promotes holistic understanding by taking into account the

complex web of connections within a system, as opposed to concentrating only on discrete

elements (Dernbach, 2009). Through global thinking, students develop a more thorough

understanding of the implications of their choices on various aspects of the system and gain

insight into the long-term effects of their actions.

The fundamental tenet of all ethics is that people are fundamentally a part of a global

community made up of interdependent parts. Human instincts may drive competition for

personal gain, but ethics motivate people to work together for the good of all (Leopold, 2015).

This profound realization highlights the importance of encouraging interconnectedness and

group efforts to address global issues in educational settings.

What this shows is that changes in educational pedagogy and subject matter over a child's

academic career greatly impact that child's ability to contribute to a prosperous and healthy

world. By incorporating global thinking into student learning standards, educators can give

students the skills, foundation, and moral framework they need to handle the complex problems

of our time.

Theoretical Framework

Global thinking helps students understand the complex systems of power, privilege, and

oppression that exist in the world (Chan & Mak, 2020). It also helps them develop empathy and

understand people's perspectives from different cultures and backgrounds. Finally, it helps them

develop the skills to think critically and analytically about global issues.

Key Concepts and Definitions

Global thinking refers to the ability to understand and analyze issues from a global

perspective. It involves recognizing the interconnectedness of the world, valuing cultural

diversity, and considering the impact of global challenges on various scales (Cloud, 2005;

Leopold, 2015).

Cognitive rigor refers to the level of complexity and intellectual challenge in educational tasks. It

involves higher-order thinking skills, such as critical thinking, problem-solving, and analysis,

which push students to engage deeply with the content and develop a deeper understanding

(Hess, 2014).

Systems thinking involve understanding the interconnectedness and interdependencies of various

components within a system. It encourages examining relationships, feedback loops, and the

broader implications of actions, allowing for a holistic understanding of complex issues

(Meadows & Wright, 2015).

Global citizenship refers to a sense of belonging and responsibility towards the global

community. It involves awareness of global issues, valuing diversity, promoting social justice,

and actively engaging in efforts to address global challenges (UNESCO, 2013).

Dr. Karin Hess, an expert in curriculum and assessment design, developed Hess'

Cognitive Rigor Matrix. The tool was designed to evaluate cognitive demands on instructional

tasks and activities (Hess et al., 2009). Educators benefit from the matrix because it allows them

to examine the cognitive rigor of tasks related to global topics and perspectives. Global thinking

requires an understanding of complex global challenges, the ability to evaluate information

critically, and a willingness to consider diverse perspectives. Using the cognitive rigor matrix,

educators can identify the level of cognitive demand required by specific tasks and ensure that

students engage in higher-order thinking processes relevant to global thinking.

The matrix assesses tasks based on four dimensions of cognitive rigor: (a) recall and

reproduction, (b) skills and concepts, (c) strategic thinking, and (d) extended thinking. These

dimensions progressively increase cognitive complexity and demand deeper levels of thinking

(Hess, 2014). The matrix was designed to aid educators in designing and assessing tasks that

align with different levels of cognitive rigor, ranging from lower-order thinking skills to higher-

order thinking skills. Hess' Cognitive Rigor Matrix consists of four quadrants that categorize

tasks based on two dimensions: Bloom's taxonomy and Webb's depth of knowledge (DOK)

(Smith & Smith, 2019). In Bloom's taxonomy, educational objectives are categorized according

to levels of cognitive complexity, while Webb's DOK framework describes cognitive demands.

The four quadrants of Hess' Cognitive Rigor Matrix are evidenced in Figures 2 and 3.

Figure 2

Hess’ Cognitive Rigor Matrix (CRM) for Reading and Listening

Note. Hess’s Cognitive Rigor Matrix for Reading and Listening. From Cognitive Rigor &

DOK Focus Area by K.K. Hess, 2009, Karin-Hess.com (https://www.karin-

Karin Hess. Reprinted with permission.

Figure 3

Hess’ Cognitive Rigor Matrix (CRM) for Writing and Speaking

Note. Hess’s Cognitive Rigor Matrix for Reading and Listening. From Cognitive Rigor &

DOK Focus Area by K.K. Hess, 2009, Karin-Hess.com (https://www.karin-

Karin Hess. Reprinted with permission.

Bloom's taxonomy is a useful instrument for evaluating global thinking because it

encompasses a variety of cognitive processes that are required for engaging with global issues.

Global thinking requires higher-order cognitive abilities, including analysis, evaluation,

synthesis, and creation. These skills go beyond basic recall and comprehension and require

students to apply knowledge, think critically, and consider diverse perspectives within a global

context.

By using Bloom's taxonomy, educators can design learning activities and assessments

that target specific levels of cognitive complexity aligned with global thinking. Assessing global

thinking using Bloom's taxonomy allows educators to evaluate students' ability to analyze global

problems, evaluate the implications of different perspectives, synthesize information from

multiple sources, and generate creative solutions to global challenges.

Bloom's taxonomy categorizes educational objectives according to their cognitive

complexity (Bloom et al., 1956). The Bloom taxonomy was developed by Benjamin Bloom and

a group of educators in the 1950s to assist educators in the design of instruction and assessment

of learning results. It consists of six levels, each representing a progressively higher level of

cognitive engagement:

Figure 4

Bloom’s Taxonomy

Note. Bloom’s levels of learning. From Bloom’s Taxonomy, by P. Armstrong, 2010, Vanderbilt

University Center for Teaching (https://cft.vanderbilt.edu/guides-sub-pages/blooms-taxonomy/ ).

CC BY-NC 4.0 DEED.

Webb's depth of knowledge (DOK) is a framework for assessing and planning instruction

that categorizes cognitive complexity, as shown in Figures 4 and 5. The DOK system was

developed by Norman L. Webb and ranks tasks based on their cognitive demand and depth of

understanding (Webb, 2002).

Figure 5

Webb’s Depth of Knowledge Levels

Note. Levels of learning based on Webb’s Depth of knowledge. From The Idea Book: Webb's

Depth of Knowledge, by the University of Saskatchewan,

n.d. (https://openpress.usask.ca/ideabook/chapter/webbs-depth-of-knowledge/ CC BY-NC 4.0

DEED.

Silver (2013) argued that educators need to integrate rigorous learning experiences that

promote critical thinking, problem-solving, and a deep understanding of global issues so that

students can cope with the challenges of the globalized world. In his article, he drew a

connection between cognitive rigor and global thinking, emphasizing the importance of rigorous

learning experiences that develop critical thinking skills and a deep understanding of global

issues. Teachers can prepare students for active participation in a globalized society by

integrating global perspectives and designing assessments that reflect cognitive rigor.

By applying Hess' Cognitive Rigor Matrix to the New Jersey student learning standards,

one can determine the alignment of cognitive rigor, which has been linked to global thinking.

Also, it is possible to identify potential gaps in the cognitive demands required by the standards.

Integrating the Three

In order to appropriately assess global thinking and the goals of the New Jersey

Department of Education's social studies goals, the teachings of Paulo Freire, John Dewey and

Benjamin Bloom were used as source materials. Paulo Freire's work focuses on empowering

students to question and transform the society they live in through critical thinking. John Dewey

believed in learning by doing. For Dewey, education is seen as a hands-on experience that

prepares students for active participation in democracy. Benjamin Bloom's taxonomy organizes

learning objectives from simple recall of facts to complex analysis and creative thinking. He

organizes these ideas through various levels.

When these approaches are blended together, students are not just taught to memorize

dates, formulas, or definitions but they are encouraged to understand why these ideas, thoughts,

concepts and skills matter in the bigger picture. For example, when students learn about history,

they are not just memorizing events; they are analyzing cause and effect, considering different

perspectives, and understanding the impact on today's world.

Teaching and learning in this style allows students to make connections to the real world.

As students explore big ideas and abstract concepts, they are encouraged to think deeply about

how these apply to their lives. Employing John Dewey and Paulo Freire's approach, teaching

through the advanced tiers of Webb's Depth of Knowledge framework enhances the significance

of educational content and boosts engagement, moving away from mere memorization of facts.

This allows students to see the links between what they learn in the classroom and what they

experience in life.

Consider how climate change, social inequality, and the pace of technological change can

be understood by students if presented as abstract, complex concepts. It can become

overwhelming, and students may see these problems as unsolvable and too complicated to

consider. By using an approach that allows students to not just learn about these issues in

isolation but to see how they connect to their lives, and if they are allowed to create solutions in

an authentic manner, they can understand how interconnected these challenges are to their lives.

When using Freire, Dewey, and Bloom as educational inspiration, students are aided in

becoming more aware of the complexities of the world, and they are also given the opportunity

to see themselves as problem-solvers. In this manner, students are empowered to contribute to

the global dialogue by asking questions, getting involved, and adding their voices.

In this manner, students are prepared to do more than simply process and be part of

academic subjects. Additionally, they are prepared to navigate and contribute to a world they are

a part of. Giving students agency by teaching critical thinking and global thinking teaches them

to lead, how to innovate, and how to make a difference so that they are ready to take on the

challenges of a global community. The goal of this holistic approach is to prepare students for

life in a fast-paced, ever-changing world, not just for state assessments but to know that

preparing them in this manner may allow them to succeed on assessments that test application,

logic, and reasoning skills.

When Freire's, Dewey's, and Bloom's ideas are integrated harmoniously in classroom

practice, they have the potential to generate a multifaceted learning experience for students,

characterized by richness, depth, and diversity. The goal here is not just to fill students' heads

with facts. In order for them to make an impact on global conversations, educators need to spark

curiosity and challenge them to see things in new ways. Students need to be engaged with

material, ideas, and concepts in order for them to ask questions about the world in which they

live. This type of thinking and learning prepares students to be stewards of the planet and

advocates for humanity. It should be noted that this type of learning requires educators to be

properly prepared and supported to deliver education at such a high level, and students need to

master these skills, not simply be exposed to them.

By blending the insights and theories of these educational philosophers, a learning

journey is being crafted that is designed to prepare students for the realities of a world which is

more connected than ever. It is about moving beyond seeing students as recipients of information

and viewing them as capable beings able to make sense of information so that a sense of global

awareness and responsibility is fostered in them. This kind of education prepares them to

navigate the complexities and seize the opportunities of a shared global community.

Methodology

A content analysis approach was used in this study's research design, which focused on

how New Jersey social studies standards for Grades K–8 assessed global thinking. Finding

patterns, themes, and relationships in textual data through content analysis, is a systematic and

objective process. The cognitive rigor matrix, created by Hess in 2009, was used in the study as

the framework for evaluating higher-order thinking in the identified standards.

The study focused on higher-order thinking because findings suggested that if teachers

purposely and persistently practiced higher-order thinking strategies— dealing in class with real-

world problems, encouraging open-ended class discussions, and fostering inquiry-oriented

experiments—there was a statistically significant improvement in critical thinking skills (Miri et

al., 2007).

• Data Collection Methods: The standards for social studies in New Jersey for Grades K–8

were examined in order to gather the data for this study. The official website of the New

Jersey Department of Education was used to access the standards document. The

standards served as the main information source for the content analysis.

• Data analysis procedures: The data analysis procedures required several steps to apply the

cognitive rigor matrix and evaluate global thinking in the NJ social studies standards. The

following steps were taken:

• Familiarization: In order to understand the document's content and structure, the

researcher carefully examined the NJ social studies standards. The standards were read

multiple times, and potential questions were noted on the margins.

• Identification of relevant standards: The researcher highlighted particular verbiage in the

standards that related to cognitive complexity and Webb's depth of knowledge.

• Application of the cognitive rigor matrix: The researcher applied the cognitive rigor

matrix, developed by Hess in 2009, to analyze the identified standards. The matrix

consists of four categories (know, apply, analyze, and evaluate) and two dimensions

(content complexity and cognitive demand). Each standard was evaluated and assigned a

rating based on its alignment with the categories and dimensions of the matrix. The

ratings were determined through a systematic coding process.

• Coding process: The coding process drew upon established content analysis methods, as

outlined by Krippendorff (2018), Neuendorf (2017), Elo and Kyngäs (2008), and Hsieh

and Shannon (2005). It involved systematically examining each standard and applying

the cognitive rigor matrix to assign appropriate codes. Specific words or phrases within

the standards that required higher-order thinking skills, critical analysis, or global

perspective were circled as indicators of global thinking.

• Application of cognitive rigor matrix: The AI model applied Hess' Cognitive Rigor

Matrix, categorizing each standard based on cognitive complexity and demand.

• Automated coding: ChatGPT systematically analyzed the standards, assigning codes

based on indicators of global thinking.

• Data interpretation: After completing the coding process, the data were analyzed. The

researchers looked for how the standards were categorized based on Webb's depth of

knowledge verbiage. This verbiage and type of categorization was tied to global thinking.

The findings were also interpreted in relation to the research objectives.

• Verification and reliability: To ensure accuracy, the AI-generated codes were reviewed

manually and compared with Hess' Cognitive Rigor Matrix by the researcher using the

framework method.

By integrating the coding procedures and principles outlined by Krippendorff (2018),

Neuendorf (2017), Elo and Kyngäs (2008), and Hsieh and Shannon (2005), the study ensured a

rigorous and systematic approach to the content analysis process. This included steps such as

familiarization, identifying relevant standards, applying the cognitive rigor matrix, coding, data

verification, and data interpretation.

Using artificial intelligence to categorize depth of knowledge (DOK) levels in social

studies standards provided multiple benefits. Artificial intelligence is an emerging technology

that is being used more and more across multiple disciplines and industries. One would suspect it

will soon become mainstream in curriculum development. Here is an explanation supported by

research-backed data:

Efficiency and Accuracy in Data Processing:

• Research insight: AI algorithms, particularly those used in advanced language models,

can process and analyze large volumes of textual data efficiently and accurately (Mayika

et al., 2022).

• Application in DOK categorization: Using AI to analyze social studies standards allows

for quicker categorization into appropriate DOK levels. The expectation is also that it can

ensure a consistent and objective analysis. This can be particularly valuable given the

scope of this research, which looks to review and categorize 264 standards.

This study used Hess' Cognitive Rigor Matrix to evaluate how the New Jersey social studies

standards for Grades K–8 prepare the students with the ability to be global thinkers. By

reviewing these standards, the researcher assessed if these standards provided the proper

guidance for educators to challenge students and engage them in thinking more critically. It

assessed higher-order thinking and critical thinking in Grades K–2, 3–5, and 6–8. The research

examined the extent and regularity with which Hess' Cognitive Rigor Matrix and Webb's depth

of knowledge are applied in these grade levels. This study evaluated whether the standards

offered sufficient opportunities for students to engage in comparative analysis, citation of

evidence, creation, and other analytical activities.

Consistency in educational standards is crucial for ensuring equitable learning opportunities

(Darling-Hammond, 2010). This assures all students have the same chances to learn and grow,

no matter where they are. Using AI and ChatGPT to look at these standards helped to accomplish

this in a more efficient manner. It takes away personal biases when looking at the various

standards. It should be noted that while it may take away personal biases, AI is not without its

own biases. While AI can help identify and reduce the impact of human biases, it can also make

the problem worse by baking in and deploying biases at scale in sensitive application areas

(Manyika et al., 2022).

Leaning on systematic analysis, Hess' Cognitive Rigor Matrix was used to assess the

standards that allowed for the identification of specific indicators of higher-order thinking

present within the New Jersey social studies standards for Grades K–8.

AI provides a standardized approach to categorizing DOK levels, reducing subjective

biases, and ensuring uniformity across different standards and curricula. Furthermore, this

method supports a data-driven, research-aligned approach to curriculum development, ensuring

that educational standards are rigorous, comprehensive, and aligned with the global thinking

goals that the New Jersey Department of Education set forth in its mission and vision. To

minimize AI bias, the researcher used factual prompts that integrate Webb's depth of knowledge

to categorize the standards.

Review of Social Studies Standards

New Jersey student learning standards for social studies (New Jersey Department of

Education, 2020) emphasized the progressive nature of the standards and their inherent

requirement for global thinking in their mission and vision statements. The mission statement

highlights the goal of social studies education to develop informed and contributing members of

local, state, national, and global communities (New Jersey Department of Education, 2020). This

distinction prioritizes an understanding beyond the local community and one that seeks to give

students agency in their understanding of their role in an interconnected world.

Global thinking is further supported by the vision statement within the NJ student learning

standards. It outlines several key attributes that students should possess:

• civic-minded, globally aware, and socially responsible;

• Exemplify fundamental values of democracy and human rights through active

participation in local, state, national, and global communities;

• Make informed decisions about local, state, national, and global events based on inquiry

and analysis;

• Consider multiple perspectives, value diversity, and promote cultural understanding;

• Recognize the relationships between people, places, and resources as well as the

implications of an interconnected global economy;

• Apply an understanding of critical media literacy skills when utilizing technology to

learn, communicate, and collaborate with diverse people around the world and

• Discern fact from falsehood and critically analyze information for validity and relevance

(New Jersey Department of Education, 2020).

The mission and vision of the NJ student learning standards for social studies clearly indicate

that they are progressive and explicitly require global thinking. By accessing and mastering the

standards, students may gain a better understanding of global issues and a greater engagement

with local and global affairs communities, think critically about diverse perspectives, and apply

their knowledge in an interconnected world.

The mission and vision of both the NJ student learning standards for social studies and the

National Council for the Social Studies (NCSS), demonstrate a shared emphasis on global

thinking and civic engagement. The NCSS (National Council for Social Studies, n.d.)

emphasizes the promotion of civic competence and the importance of educating students who are

committed to democratic values. They speak to the goal of having students possess knowledge

about their community, nation, and world, as well as the ability to apply inquiry processes,

analyze data, collaborate, make decisions, and solve problems. The NCSS also acknowledges the

significance of global connections and the need for students to understand diverse global

societies.

Both the NJ student standards for social studies and the NCSS standards aim to promote

civic engagement, global awareness, and an understanding of the interconnectedness of our

world. Developing informed and active citizens is emphasized in both sets of standards. They

both address the importance of citizens being able to analyze information critically, consider

multiple perspectives, and contribute to their communities, both locally and globally.

John Dewey

In today's fast-moving world, technology is changing daily life quicker than ever.

Technological advancements affect science, how businesses work, and how individuals interact

with the world on a daily basis and throughout their day. The world is getting smaller because

individuals can talk to each other in real time, share news, and get information any time of the

day with very minimal effort. However, unlike in the recent past, humans are no longer simply

receiving information now; everyone has the ability to create their own content, not just look at

what others have made. Technology companies have not just made technology easily accessible,

but they are also monetizing individuals’ interactions with it. To succeed in this connected world,

individuals must become tech-savvy and adaptable.

In the information age, knowledge and skill are power, but creativity and adaptability

reign supreme. The advancements and changes will inevitably lead to a greater need for

sophisticated knowledge and skills in the workplace. This means that there will be a demand for

students to learn how to process information in order to develop more advanced skills at a

younger age. Thus, American students are reaching a critical point in their education where they

can only absorb a certain amount of advanced content before their cognitive limits are reached.

To address this, it is necessary to modify the approach. Instead of trying to expand the

capacity of knowledge, like enlarging a cup, which ultimately has its limits, the focus should be

on changing the content within the cup. Altering the content can increase the density of

knowledge without overwhelming students. John Dewey was an influential American

philosopher, psychologist, and educational reformer who greatly contributed to the field of

education. His work emphasized the importance of education as a means of fostering democratic

values and active participation in society. Dewey believed that education should be student-

centered and focused on the needs and interests of learners rather than being solely content-

driven.

He believed education should be experiential and hands-on, allowing students to engage

in practical and meaningful activities that promote critical thinking, problem-solving, and

inquiry-based learning (Dewey, 2015). As a progressivist, Dewey emphasized the integration of

academic subjects with real-life experiences, as well as the development of social skills and

moral character, as seen in Figure 6.

Figure 6

John Dewey’s Theory of Structure

Note. John Dewey’s theory of learning. From John Dewey's Theory, by P. Main, 2023, Structural

Learning.). (https://www.structural-learning.com/post/john-deweys-theory). Structural-Learning

According to Ornstein, in progressivist thought, these skills include problem-solving and

scientific methods. Schools should nurture cooperation and self-discipline and transmit the

society's culture. Because reality is constantly changing, Dewey saw little need to focus on a

fixed body of knowledge. Progressivism emphasizes how to think, not what to think (Ornstein &

Hunkins, 2018). Progressivism, as argued by Sahu (2002), asserts that teaching should evolve

alongside societal changes, emphasizing adaptability to new advancements rather than adhering

rigidly to past methodologies. Here, learning is based on children's innate curiosity in trying to

make sense of the world by accessing a cognitively rigorous curriculum.

Considering the limitations on the quantity of content that can be covered within a school

year, it is essential to prioritize learner-centered approaches to education more than ever before.

Today's kindergarteners, for example, will not solely need to memorize facts and details or recall

the chronological order of U.S. Presidents. Instead, they will need to possess the skills to adapt

effectively to rapidly changing circumstances and to generate innovative solutions for problems

that are currently unknown to the world.

This shift highlights the importance of cultivating higher-order thinking skills and global

thinking abilities. By fostering critical thinking, problem-solving, and creativity, students may be

better equipped to navigate an ever-evolving world. Education should give students agency over

what they are learning. It should provide them with opportunities to think critically, analyze

complex situations, consider multiple perspectives, and apply their knowledge to real-world

challenges. By placing the learner at the center, schools can provide opportunities for students to

perform more effectively in a future world with more dynamic demands.

Researchers believe that students learn best by diving into content that is tailored to them,

and students are expected to learn in a group and from each other in a constructivist manner.

Learning happens best when it is done by solving real-world problems and using new ways to

find information. This real-world application is where students are able to apply what they learn

in a more authentic manner. This student-centered approach focusing on real issues that need

creative solutions equips students with the necessary skills to think critically and logically.

Learning should take place for the purpose of understanding and making sense of the world in a

dynamic and interactive manner, which happens at the higher levels in Webb’s depth of

knowledge framework. Education should not be seen as a tool to simply get a job but as an

opportunity to develop the necessary skills to contribute to the world in a purposeful and positive

manner. This means education needs to be more about active participation and collaboration

rather than just memorizing facts. Learning that involves recalling and remembering occurs at

the lower levels of Webb's depth of knowledge framework, where critical thinking is less

developed.

The researcher sees education as the most powerful tool in shaping a future where

learning and society change and grow together for the betterment of the planet and all its

inhabitants. By embracing this change, education becomes a force for making a difference. When

education is interactive and constructive, it helps students find their own understanding of what

learning means. This approach not only transforms students but also prepares them to tackle the

complex challenges of sustainability.

Paulo Freire

According to Freire, critical consciousness and transformative education are tools that

can be used for liberation and social change (Freire, 1970). A traditional education system treats

students passively by transferring knowledge to them. Instead, Freire advocated for an approach

to education that encourages critical thinking, dialogue, and active engagement with the world,

which requires a cognitively rigorous curriculum.

Critical thinking is an essential component of Freire's educational philosophy. It involves

the ability to analyze, evaluate, and question information and ideas in a systematic and reflective

manner. Rather than accepting information passively, critical thinkers engage in deep reflection,

consider multiple perspectives, and assess the credibility and relevance of the information

presented to them, which places it at Bloom’s highest level, “evaluate” “create” and Webb’s

DOK Level 3 and Level 4.

Freire argued that education should not simply transmit established knowledge and

societal norms but ought to enable students to critically examine and challenge them (Mahmoudi

et al., 2014). Students gain agency over their learning when they make sense of the information

they are introduced to. By gaining agency through critical thinking, students can also have a

deeper understanding of the world. They can identify social injustices, take action to bring about

positive change and work toward making the world a better place.

Through critical thinking, education becomes a dynamic process that engages students in

questioning the status quo. It provides them the skillset to identify and challenge oppressive

structures while also engaging in dialogue and collaboration with others. As students engage

actively with the world, they develop critical consciousness. They become aware of social and

political issues. They recognize power dynamics and imagine possibilities for social change.

Living in a global society and recognizing systems of oppression requires a global perspective.

This global perspective leads to an understanding of interconnectedness, interdependence, and

the complex issues that go beyond national borders. It involves recognizing global challenges,

valuing diversity, and considering multiple perspectives (Mundy, 2002). People who engage in

global thinking consider issues such as social justice, human rights, and environmental

sustainability beyond their immediate context.

Pedagogy of the Oppressed by Paulo Freire and global thinking are closely linked

because both emphasize critical consciousness, dialogue, and social transformation, all of which

need to be supported by standards that promote global understanding. Freire's approach

challenges oppressive systems and encourages individuals to critically analyze their social

reality, question dominant narratives, and work toward social justice (Freire, 1970). By doing so,

it encourages an understanding of international social, cultural, and political issues.

Global citizens can be nurtured by cultivating critical consciousness and a deeper understanding

of power dynamics through the Pedagogy of the Oppressed. It aligns with the principles of

global thinking by encouraging individuals to critically examine and challenge existing

structures and to promote social equality and solidarity on a global scale (Mundy, 2002).

Pedagogy of the Oppressed and global thinking are interconnected and mutually reinforcing

concepts that promote critical awareness, social justice, and active engagement with the world.

Overview of Social Studies Standards

In 2020, the New Jersey Department of Education released the new student learning

standards for social studies (NJSLS-SS). NJSLS-SS were progressive and forward-thinking; they

were created with three overarching standards: U.S. History: America in the World, World

history/global studies, and active citizenship in the 21st century. These new standards have a

greater focus on depth over breadth, with a strong emphasis on developing learners who are

civic-minded, globally aware, and socially responsible (New Jersey Department of Education,

2020). The mission and vision sections provide a vivid description of what NJ social studies

aspires to foster and provide for student learners. The NJ DOE encouraged teachers that the

content did not need to be linear. The central goal was to create interdisciplinary units that were

evidence-based action and solution-oriented (Hanna, 2021). Many of the performance

expectations deliberately called for collaboration, deliberation, reflection, and questioning key

attributes within social-emotional learning. These also emphasize higher-order thinking skills

and support students in building those critical SEL skills (Hanna, 2021).

The Case for Integrating Global Thinking in Social Studies Standards

PISA, which stands for the Programme for International Student Assessment is a

worldwide study by the Organisation for Economic Co-operation and Development (OECD) in

member and non-member nations intended to evaluate educational systems by measuring

students' performance in mathematics, science, and reading literacy every three years. Their

framework for global student learning outcomes offers a comprehensive process for integrating

global perspectives and skills in education. PISA offers a framework for “global competence

assessment which measures students’ capacity to examine local, global and intercultural issues,

to engage in open, appropriate and effective interactions with people from different cultures, and

to act for collective well-being and sustainable development.” (OECD, 2020). According to the

framework, students need to understand key ideas like cultural diversity, global systems and

issues, and global citizenship to become globally competent (OECD, 2020). These ideas give

students a foundation on which to explore and comprehend the complexity of the world. In

addition to offering a set of fundamental questions, the framework also provides a set of

directives for researching and examining global issues (OECD, 2020). The framework

encourages students to examine world issues, consider multiple viewpoints, and broaden their

knowledge of the world. Additionally, PISA offer a codebook developed in partnership with

Global Cities, Inc. Global Cities, Inc. is part of a program with Bloomberg Philanthropies. They

offer a codebook which is a “tool for evaluators and teachers interested in determining how well

students are demonstrating global learning in any program or classroom” (OECD, 2020).

They highlight the importance of incorporating a variety of viewpoints and resources. They

speak to the value of fostering collaborative and inquiry-based learning opportunities (OECD

2020).

Reviewing the framework was critical as it included a crucial component that evaluated

global competence (OECD, 2020). It suggested several methods, such as performance-based

evaluations, projects, and reflections, to gauge students' knowledge, abilities, and attitudes in

relation to global education. These skills align directly with Hess' Cognitive Rigor Matrix,

specifically on the higher order side of the matrix. When students are encouraged to examine

world issues and consider multiple viewpoints, as suggested by the framework, they are

engaging in higher-order thinking skills. These skills are the same as detailed in Hess' Cognitive

Rigor Matrix and Webb's DOK. The framework's emphasis on stimulating, culturally sensitive

learning environments, incorporation of diverse viewpoints, and fostering collaborative and

inquiry-based learning directly aligns with the higher levels of cognitive rigor and depth of

knowledge that these educational frameworks advocate.

To ensure the integration of global education, the framework also acknowledges the need for

supportive systems and policies (OECD, 2018), which the researcher used to provide further

recommendations. To advance global education, policymakers, and academic leaders are urged

to create opportunities for professional growth, provide adequate funding, and form alliances. In

general, the framework for global education provides policymakers and educators with a

roadmap for incorporating global perspectives, abilities and understanding into the curriculum.

By incorporating global perspectives into the social studies curriculum, students can better

understand and address global issues. Additional scholarly sources further support the imperative

for a global education. These findings align with the framework proposed by the OECD (2018),

emphasizing the need for comprehensive systems and policies to support the integration of

global perspectives into education.

• Examining global interconnections: This framework suggests there should be a focus on

helping students understand how the world is interconnected (OECD, 2013). Educators

should guide students in exploring the connections and influences between societies,

cultures, economies, and environments (OECD, 2013).

• Analyzing global systems and challenges: Students should be encouraged to analyze

global issues, such as economic interdependence, environmental sustainability, political,

and social justice (Cornell University, n.d.). Through discussion and investigation,

teachers can help students evaluate these complex issues critically.

• Investigating cultural diversity: The New York State Education Department puts a focus

on cultural responsiveness and states that, “having high expectations and delivering

rigorous instruction for all students regardless of identity markers, including race, gender,

sexual orientation, language, ability, and economic background,” creates a welcoming

and affirming environment. By exposing students to a variety of cultures, traditions,

beliefs, and practices, teachers can inspire their students to respect and value different

points of view and ways of living (NYSED, n.d.).

• Developing global citizenship: Is stressed as part of the Universal Declaration of Human

Rights, first drafted in 1948 after World War II. This includes understanding one's rights

and responsibilities in the global community (Israel, 2015). According to the framework,

developing global citizenship entails being aware of one's obligations to the world

community (Israel, 2015).

In line with these perspectives, it is evident that the inclusion of global education in the

curriculum is not merely a pedagogical option but a necessity for preparing students to thrive in

an interconnected world. For this reason, policymakers and academic leaders must prioritize

initiatives that promote global competence, including professional development opportunities,

adequate resource allocation, and collaborative partnerships. Through intentional integration of

global thinking within the social studies curriculum, educators can empower students to critically

engage with complex global issues and contribute meaningfully to a more interconnected and

interdependent world.

Role of AI in Education

The development of artificial intelligence has had a significant impact on modern society.

Modern society is changing as a result of it. It is making a significant impact in businesses and

individuals' daily lives. AI can make work way more efficient and help with new ideas. AI

allows work to get done better and quicker. Writing an email or letter for mass distribution in the

past could have taken hours. Now with the proper prompt it can be done in minutes.

Analyzing articles could take days or weeks; now, it can be done in minutes. It is estimated that

80% of the world’s data is unstructured, and a significant proportion of them are text-heavy. In

order to find insights among unstructured text, text analytics techniques are employed to

transform the text into data that can be used for further analysis (Papadopoulos & Charalabidis,

2020). The researcher chose to integrate AI in order to extract information from the dataset more

quickly and efficiently.

Extraction refers to the process of identifying and isolating important pieces of data from

a larger dataset. In this case, AI extracted specific verbiage from the New Jersey social studies

standards for Grades K–8 that is aligned with Hess' Cognitive Rigor Matrix and Webb’s depth of

knowledge framework in order to identify cognitive complexity within the standards. This

allowed AI to evaluate student learning outcomes. These results were then validated by the

researcher who applied the framework method to review the findings.

Chapter 3: Methodology

In 2020, the New Jersey Department of Education updated the New Jersey student

learning standards for social studies with the mission that: Social studies education provides

learners with the knowledge, skills, attitudes, and perspectives needed to become active,

informed, and contributing members of local, state, national, and global communities.” These

goals align with the United Nations' global sustainability goals, which state, “Ensure inclusive

and quality education for all and promote lifelong learning, which includes global citizenship as

one of its targets, (United Nations, n.d.). As a result, students can gain a deeper understanding of

the world by learning about world events through global citizenship. While the goals created by

the New Jersey Department of Education are progressive and forward-thinking, do they set forth

the framework for students to develop the competencies to become global citizens?

Research Questions

The study was guided by the overarching research question, which sought to investigate

whether the 2020 New Jersey student learning standards for social studies in Grades K–8 aligned

with the cognitive rigor matrix proposed by Hess. The research question sought to know if the

standards effectively provided students with the necessary framework for developing the

competencies required for global citizenship, specifically by examining the presence,

complexity, and frequency of higher-order thinking opportunities. In this study, the researcher

looked at how the language used in these standards matched Hess' definition of higher-order

thinking. The researcher assessed how cognitively complex the standards were and how many

chances they gave for higher-order thinking using Hess' Cognitive Rigor Matrix The following

additional questions served as a guide for this research:

• Research Question 1: In what way(s) does the language found on the 2020 New Jersey

student learning standards for social studies in Grades K–8 compare to Hess' Cognitive

Rigor Matrix definition of higher-order thinking?

• Research Question 2: How cognitively complex are the 2020 New Jersey student learning

standards for social studies in Grades K–8 as defined by Hess' Cognitive Rigor Matrix?

• Research Question 3: How frequently do the 2020 New Jersey student learning standards

for social studies in Grades K–8 include higher-order thinking opportunities as identified

in Hess' Cognitive Rigor Matrix?

Research Design and Methodology

The study used a mixed-methods approach. The qualitative aspect involved conducting a

qualitative content analysis of the language used in the New Jersey student learning standards for

Grades K–8. The content analysis was meant to better understand the standards' language,

higher-order thinking requirements, and alignment with the desired outcomes for global

thinking. Qualitative content analysis is a research approach for the description and interpretation

of textual data using the systematic process of coding (Assarroudi et al., 2018). Additionally,

“qualitative research allows asking questions that cannot be easily put into numbers to

understand human experience. Getting at the everyday realities of some social phenomenon and

studying important questions as they are really practiced helps extend knowledge and

understanding” (Cleland, 2017, p. 61).

Due to the nature of this study, the researcher chose to combine the framework method

and the Mayring step model. In doing so, the researcher was able to leverage the structural

benefits of the framework method, which is used for data organization and thematic analysis.

The Mayring step model allows for detailed textual analysis. This approach is particularly useful

in mixed-methods research, where both broad thematic insights and detailed text interpretations

are based on a predetermined coding scheme (Mayring, 2000). The framework method was used

to help organize the standards and for the researcher to become more familiar with the dataset,

the New Jersey social studies standards for Grades K–8. The standards were organized and

shared with AI so that a detailed analysis could be applied by grade level. This allowed the

researcher to more easily categorize the dataset, understand the data, and find patterns. The

standards were inputted into AI, starting from Grades K–2 first, then 3–5 and finally 6–8.

The researcher coded the dataset twice with AI, on two separate days on two separate

computers and then used the Mayring step model to refine the thematic analysis as well as

enhance validity and reliability. The Mayring step model allowed the researcher to review the

standards more deeply using Webb’s depth of knowledge framework. When there were

disagreements between AI and the researcher, the researcher, as the educator, made the final

decision on the categorization of the standards, as shown in Figure 7.

Figure 7

Framework Method and Mayring Step Model Process

Note. Synthesis of the Mayring Step Model and Framework Method. Adapted from “Using the

framework method for the analysis of qualitative dyadic data in health research” by N. Collaço,

R. Wagland, O. Alexis, A. Gavin, A. Glaser, & E. K. Watson, 2021, Qualitative Health

Research, 31(8) (https://doi.org/10.1177/10497323211011599) and “Qualitative Content

Analysis” by P. Mayring, 2000, Forum Qualitative Sozialforschung

1(2) (https://www.researchgate.net/publication/215666096_Qualitative_Content_Analysis). Both

articles licensed under CC BY 4.0.

The quantitative part of the research highlighted the use of artificial intelligence (AI) and

natural language processing (NLP) as essential tools for data analysis. These advanced AI

technologies were vital for evaluating the New Jersey student learning standards and aligning

them with Hess' Cognitive Rigor Matrix framework, Webb's depth of knowledge, and Bloom's

taxonomy.

This research utilized AI tools, particularly the Chat GPT model, to categorize and

organize the language within the standards. Through prompting, the AI model was trained to

identify specific keywords, phrases, or concepts related to different levels of cognitive

complexity, as defined by Hess' Cognitive Rigor Matrix. As part of the methodology, the

prompts were provided to replicate the study. These helped to better understand how AI

connected the identified keywords, phrases, or concepts to established cognitive frameworks,

such as Hess' Cognitive Rigor Matrix, Webb's depth of knowledge, and Bloom's taxonomy. This

mapping process provided a structured way to assess the standards based on their alignment with

these frameworks, as shown in Figure 8.

Figure 8

Sequence Mapping for Coding Standards

Note. Sequence Diagram of AI-Assisted Analysis and Classification of Educational Standards.

This diagram illustrates the three-part process of input and interpretation of raw educational

standards, analysis and classification. Own Work.

Part 1 required the researcher to input the prompt and then upload the data and standards

into AI, ChatGPT. The standards were copied and pasted right into AI, allowing it to read and

begin interpreting them. Part 2 involved ChatGPT analyzing the cognitive demands and

classifying the standards based on Webb’s depth of knowledge levels. Finally, in Part 3 ChatGPT

returned the standards in a table that included the standard as well as the Webb’s depth of

knowledge Level, 1–4.

The AI and NLP were used because of their efficiency and consistency in the analysis. AI

was able to quickly and consistently process large amounts of text data and reduce the risk of

human bias and error. It should still be noted that bias can creep into algorithms in several ways.

AI systems learn to make decisions based on training data, which can include biased human

decisions or reflect historical or social inequities, even if sensitive variables such as gender, race,

or sexual orientation are removed (Manyika et al., 2022). Incorporating AI and NLP into the

research design represented an innovative approach to educational assessment. By using this

method, the New Jersey student learning standards could be analyzed with greater objectivity

and reliability. To ensure the validity of the study, the researcher used the framework method in

conjunction with the Mayring step model to review the findings.

In education, the strength of quantitative research is that the findings are derived from

finding exact facts; that is, the same research methods are applied, and the results are generalized

(Hara, 1995). In contrast to mono-method research, mixed-methods research often produces

superior results due to its methodological pluralism and eclecticism (Johnson & Onwuegbuzie,

2004).

Using the mixed methods approach allowed the research to identify categories, themes,

and patterns related to the distribution and frequency of high-order thinking in the 2020 New

Jersey student learning standards for social studies in Grades K–8, based on Hess' Cognitive

Rigor Matrix. The mixed-methods approach allowed for comparing, analyzing, and categorizing

the language of higher-order thinking found within the standards.

Webb's depth of knowledge (DOK) and Hess' Cognitive Rigor Matrix are tools used

together to help understand the cognitive demands and complexity of educational standards.

Educational materials are designed using both frameworks to ensure they align with the intended

cognitive rigor level. Here is a detailed explanation of how these two frameworks can work

together.

Webb's Depth of Knowledge (DOK)

Webb's DOK framework, as shown in Figures 8 and 9, categorizes tasks and standards

into four levels based on the cognitive complexity required to complete them:

DOK Level 1 (recall and reproduction): These tasks are considered to be the simplest.

They ask students to recall information or reproduce facts or procedures. It doesn’t involve much

thinking. Asking a student the title of a book or at what temperature water boils would fall into

this level.

DOK Level 2 (basic application of concepts and skills): These require students to apply

knowledge and skills to solve simple problems or perform routine tasks. Asking a student to

contrast or classify would fall into this level.

DOK Level 3 (strategic thinking and complex reasoning): These tasks demand more

complex thinking from students and require them to use reasoning skills. Here, students are

expected to analyze, synthesize, and draw conclusions from information they are presented.

DOK Level 4 (extended thinking and complex reasoning): These tasks are considered to

represent the highest cognitive demand. They require students to think critically, analyze

multiple sources of information, make connections across subjects. These types of tasks allow

student to think more authentically and develop their own idea.

Figure 9

Webb’s Depth of Knowledge

Note. Depth of knowledge learning levels. From Striving for Higher-Order Thinking and Depth of

Knowledge, by T. Watanabe, 2013, wwwatanabe (https://gcci.uconn.edu/2017/11/14/building-

classroom-dialogue-using-webbs-depth-of-knowledge/) CC BY-NC 4.0 DEED.

Figure 10

Webb’s DOK Question Stems

Note. Levels of learning based on Webb’s Depth of knowledge. From Webb's Depth of Knowledge, by P.

Main, 2023, Structural Learning. (https://www.structural-learning.com/post/webbs-depth-of-

Hess' Cognitive Rigor Matrix

Hess' Cognitive Rigor Matrix categorizes educational standards and tasks into four levels

similar to Webb's DOK but uses slightly different terminology which focus more on the product

of the task.

Level 1 (recall): At this level, the focus is on whether students are able to remember

factual information. It involves tasks like remembering dates of events.

Level 2 (skill/concept): At this level the focus in on if students are able to apply

knowledge and skills to solve problems or perform tasks. An example would be tasks

requiring students to understand cause and effect.

Level 3 (strategic thinking): At this level, there is a demand for more complex cognitive

skills. Students need to analyze information and think critically to solve problems.

Level 4 (extended thinking): This is the level with the highest cognitive demand. Students

are having more complex thoughts.

Using Webb's DOK and Hess' Cognitive Rigor Matrix Together

While the two frameworks complement each other and both frameworks have four levels.

Webb's Depth of Knowledge has a greater focus on the cognitive processes involved in a task

and emphasizes how students engage with and approach the task. Hess' Cognitive Rigor Matrix

on the other hand tends to have a greater focus on the expected outcome of the task. It focuses on

what students should be able to demonstrate as a result of their learning process.

It should be noted that there is a wide range of perspectives and discourses regarding what the

term global citizenship means. Its definition can be seen as undefined and even abstract.

Nussbaum (2017) argued that global citizenship requires an ability to think as a citizen of the

whole world, not just some local region or group. This view of global citizenship is closely

aligned with that of (Haigh, 2014), whose aspiration and vision is that future graduates (who they

remind us will be both our future leaders and neighbors) will be able to solve the problems of the

future, to care for those who cannot care for themselves, to defend the social values that we

believe in, to protect the environment for those who will follow and to help set the world on a

sustainable course. This type of social consciousness and problem-solving ability requires

intercultural intelligence, the ability to solve problems, comprehensive communication, and

compassion, which is achieved through cognitive engagement with complex rigor, which aligns

more readily with Hess' Cognitive Rigor Matrix framework.

Reliability Assurance in Qualitative Data Analysis

Reliability is a cornerstone of rigorous research, ensuring the consistency and

dependability of data analysis procedures. The qualitative phase of this research looked at the

content language of the 2020 New Jersey student learning standards for social studies in Grades

K–8. The dataset were also run through AI twice. In both cases, the researchers run the three

different sessions to input the standards by grade level. The researcher then integrated the

framework Method and the Mayring step model to analyze the data.

Role of ChatGPT

Artificial intelligence was used to perform the content analysis to ensure reliability in

qualitative analysis. The research employed the use of ChatGPT. ChatGPT played an important

role in coding the standards using Hess' Cognitive Rigor Matrix and Webb's depth of knowledge.

Here's a breakdown of ChatGPT's role in the process:

Content Analysis and Categorization

ChatGPT was employed to analyze the text of the New Jersey social studies standards for

Grades K–8. Its primary task was to read, understand, and categorize each standard based on

Hess' Cognitive Rigor Matrix and Bloom’s taxonomy. The AI model was able to identify key

phrases and concepts within the standards that corresponded to different levels of cognitive

demand, from basic recall to higher-order thinking skills.

Efficiency and Scalability

The volume of standards and the complexity of their content required an efficient tool for

analysis. Using ChatGPT allowed for a quick and seamless process that reviewed all pages of the

standards in a small amount of time. Using ChatGPT allowed the standards to be analyzed

objectively, helping to avoid mistakes and unfairness. This helped to stay consistent, avoids

biases, and unfair slants. The software stuck to a strict set of rules from the cognitive rigor

matrix, so every standard gets judged by the same high bar.

Coding Reliability

To further strengthen the reliability of the qualitative analysis the researcher assessed the

results of ChatGPT. In these checks, the researcher independently coded the data and then

compared the results with that of ChatGPT. Errors were analyzed and documented. Where cases

existed and there was a significant departure from AI coding and researcher opinion,

discrepancies were analyzed and reviewed, and compared to established literature.

Transparent Reporting of Methodology

To assure transparency, the prompts inputted into AI were provided to allow replication

of the study.

Additionally, the researcher clearly documented the version of the AI tool used, how the data

were inputted, and any limitations or challenges encountered.

As a single researcher conducting coding for this research, ensuring inter-coder reliability

was important to ensure the validity of the finding. For this reason, the following is a step-by-

step guide on how the researcher reviewed the AI ChatGPT findings.

Clear Coding Guidelines

• ChatGPT was provided with the prompt to code the standards. The prompts and outputs

were shared in Appendix B of this research.

• The standards were assessed by ChatGPT twice using the same prompts.

• The standards were assessed in separate sessions. First, it assessed starting from Grades

K–2, then 3–5, and finally 6–8.

• The researcher provided the findings of all coding findings in this Appendix

• The researcher made the final determination on the classification.

Codebook Creation

• The researcher used Webb’s depth of knowledge graphic by Tracy Watanabe and depth

of knowledge – Descriptors, Examples, and Question Stems for Increasing Depth of

Knowledge in the Classroom Developed by Dr. Norman Webb and Flip Chart developed

by Myra Collins to review AI, ChatGPT findings.

• Discrepancies were highlighted. The researcher also provided a reason for the

discrepancies.

Reflection and Revision

• The researcher reflected on and reviewed coding decisions periodically.

• The researcher consulted the coding guidelines when unsure.

Code Comparison and Documentation

After the researcher completed the coding, a reliability check was conducted.

Discrepancies were documented. Throughout the coding process, challenges and errors were

documented and reviewed.

Validity

Enhancing the validity of a study that utilizes AI, such as ChatGPT, for analyzing

educational standards involved several key strategies. These strategies were designed to ensure

that the research findings accurately reflected the reality they intended to represent. This study

used multiple data sources, methods, or theories to cross-verify the results. This study combined

AI analysis and a human conducting manual coding. Additionally, the researcher used multiple

prompts to introduce AI to Hess' Cognitive Rigor Matrix Webb’s depth of knowledge, and

Bloom’s taxonomy. AI tested the standards with the same prompts to provide consistency.

Participants and Sampling

This study did not use any participants or population sampling.

Data Sources and Collection

The data gathered were retrieved from a public website containing the New Jersey

student learning standards (NJSLS) in social studies, which is reviewed and revised every 5

years. The 2020 NJSLS in social studies was adopted by the State Board of Education on June 3,

2020 (New Jersey Department of Education, 2020). The data can be collected at any time. As the

NJSLS was developed in 2020, new standards will not be developed until 2025. The data were

copied and pasted directly from the New Jersey Department of Education and pasted onto a

Google Doc before being inputted into the AI program. The standards are posted on the New

Jersey Department of Education website by grade level. It is provided in Grades K–2 (54

standards), 3–5, (100 standard), and 6–8 (109 standards). Figure 10 below summarizes the

standards being analyzed by grade.

Figure 11

Social Studies Standards by Grade Levels

Note. Distribution of the NJ Student Learning Standards for Social Studies across various grades

between K-8. Own work.

Data Analysis

In order to analyze the New Jersey student learning standards in social studies, the

researcher used Hess' Cognitive Rigor Matrix. Bloom's taxonomy correlates cognitive

complexity with Webb's depth of knowledge, and Hess' Cognitive Rigor Matrix correlates task

actions with Webb's depth of knowledge. The student's cognitive level and the type of task they

are expected to perform can, therefore, be examined for each standard, allowing multiple

perspectives to be explored.

Educational standards are the learning goals for what students should know and be able to

do at each grade level (What are educational standards? What are educational standards?

(Common Core State Standards Initiative, n.d.). Assessing the standards allowed the researcher

the opportunity to identify the type of language within the standards, its cognitive complexity,

and its frequency. It allowed for (a) deductive coding analysis, (b) assessment levels of rigor

using Hess' Cognitive Rigor Matrix, and (c) assessment frequency using Hess' Cognitive Rigor

Matrix.

Additionally, the Hess' Cognitive Rigor Matrix links the cognitive complexity of Webb’s

depth of knowledge and the actions of tasks categorized in Bloom’s taxonomy. This allowed for

each standard to be analyzed not only for the type of task that was being asked of the students

but also for the level of cognition students are required to utilize, allowing each standard to be

examined from multiple perspectives

Validity and Credibility

Assuring the validity and credibility of the research is paramount. As Merriam (2009)

pointed out, validity and reliability are closely related. Internal validity from a traditional

perspective rests for some on the assumption that a study is more valid if repeated observations

in the same study or replications of the entire study produce the same results. In this logic, truth

is established by repetition, but people, measurements, and observations are all subject to error

(Merriam, 2009). In terms of research design, Merriam (2009) suggested that reliability is a

result of repeatedly studying a single reality and yielding the same results. Research reliability

refers to the reproducibility of results. If repeated, will the results be replicated? Human behavior

is never static, which makes reliability in the social sciences difficult. In order to ensure the

validity of the research, a review of coding methods from other studies was included. The

research also included two analysts in coding each of the standards and then comparing their data

and findings, increasing inter-rater reliability. All standards were coded based on Hess' Cognitive

Rigor Matrix coding protocol. All prompts used with ChatGPT were provided.

Limitations

The study was designed to examine the New Jersey social studies standards in Grades K–

8. The analysis did not cover the standards for high school. Higher-order thinking and cognitive

complexity are measured in multiple ways. A clear, definitive connection between cognitive

complexity, higher-order thinking, and global thinking has not been established.

While there are strengths for using AI, in this case ChatGPT, to analyze the New Jersey

student learning standards for social studies, it was also important to understand the limitations.

These limitations could impact how the standards are interpreted. AI is proficient at repeatedly

carrying out the same task, but if we want any adjustments or improvements, we must manually

alter the codes. (Duggal, 2023) Additionally, AI systems can inherit biases present in their

training data. These biases could influence how the standards are analyzed and categorized,

potentially leading to skewed results (Mayika et al., 2022). The biggest concern was that while

the researcher understood the prompts being used to initiate the coding, there was no

transparency about the AI algorithm being used.

Chapter 4: Data Analysis and Results

This study was guided by the overarching question: In what way(s) does the language

found on the 2020 New Jersey student learning standards for social studies in Grades K–8

compare to Hess' Cognitive Rigor Matrix definition of higher-order thinking? In order to

understand the findings of this mixed-methods study, which used artificial intelligence (AI) to

assess the New Jersey student learning standards for social studies in Grades K–8, it was

important to understand how AI was used and how the research was standardized. The research

reviewed 263 standards. The standards were inputted into AI twice, on two different days, using

two different computers. These results were then reviewed by the researcher using the

framework method as well as the Mayring step method.

The researcher first used AI to assess the 263 standards, followed by the framework

method and Mayring step model as coding tools for the analysis of qualitative data to review the

AI findings. The framework method has been used since the 1980s (Gale et al., 2013b). The parts

of the framework are as follows:

Familiarization: The aim of this stage of framework analysis is to get to know the data

extensively (Parkinson et al., 2015).

Identifying a framework: The aim of this stage of framework analysis is to organize

data in a meaningful and manageable way for subsequent retrieval, exploration and examination

during the final mapping and interpretation stage (Parkinson et al., 2015).

Indexing: This involves systematically applying the framework to each interview

transcript (Parkinson et al., 2015).

Charting: The aim of the charting stage is to organize the data into a more manageable

format to facilitate data analysis in the next stage of framework analysis (Parkinson et al.,

2015).

Mapping and interpretation: The aim of this stage of framework analysis is to move

beyond data management towards understanding it (Parkinson et al., 2015).

The researcher first prompted AI, ChatGPT, with the commamd to begin the coding

process. It required that the researcher prepare ChatGPT with the “mindset” of the work it would

perform as well as how the work would be performed. The first prompt was,

You are conducting doctoral research on the New Jersey social studies standards. I will

give you about 10 standards at a time, copied from the New Jersey Department of

Education website. You will use Hess' Cognitive Rigor Matrix to label the standards by

level based on the verbiage or task demand. Specifically, you will use Hess' Cognitive

Rigor Matrix (Reading CRM) and (writing-speaking CRM), applying Webb's depth of

knowledge levels to Bloom's cognitive process dimensions. You will create a table with

column one for the standards and column two for Webb's depth of knowledge classification.

Do you understand?

The complete interaction between the researcher and ChatGPT is shared in Appendix B.

This prompt was able to define the framework by asking Hess' Cognitive Rigor Matrix

framework. This framework contains the cognitive skills being focused on to identify higher-

order thinking within the New Jersey student learning standards for social studies in Grades K–8.

Using this framework allowed AI (ChatGPT) and the researcher to have a clear definition of

cognitive skills, which would ensure consistent application. The researcher then inputted

approximately 10 of the New Jersey learning standards into ChatGPT at a time. The researcher

also started a new session with the same prompt three times. One new session was used for each

of the grade levels.

In the context of using ChatGPT, a new session generally refers to starting a fresh

interaction with the model without any prior context or history from previous interactions. Each

session is isolated, meaning that ChatGPT does not remember the conversation from past

sessions. When you start a new session, it is as if you are interacting with ChatGPT for the first

time, with no memory of previous questions, answers, or any other information shared in earlier

sessions (OpenAI, 2024). In the first session, it reviewed the standards from Grades K–2; in the

second, it was Grades 3–5, and finally, from Grades 6–8 standards. Three different sessions were

used in case there were errors or disruptions so that a new session would not have to be initiated

for the entire coding process. This would have been time-consuming and would not have

improved the results. Inputting the standards approximately 10 at a time allowed AI to

familiarize itself with the standards while getting a general sense of the content and structure.

The prompt to use Hess' Cognitive Rigor Matrix framework allowed AI to categorize the

cognitive skills that are within the standards based on Webb’s depth of knowledge from Level 1–

4. Once the standards were entered the coding process commenced and AI read through and

created a table as instructed which allowed the researcher to see the standard with the

corresponding Webb’s depth of knowledge level.

The consistency was done by the researcher by adding approximately 10 standards at a

time without any additional prompts or information to assure the same process was used for the

entire coding process. No adjustments were necessary. To analyze the data, the researcher then

inputted the prompt, “What is the frequency of each level. I will input all levels.” The levels

were copied from the table and inputted one by one.

This allowed AI to calculate the frequency of Webb’s depth of knowledge levels for the

entire Grades K–2 standards, then Grades 3–5, and finally Grades 6–8. The following Table 12

shows how steps one through six were implemented.

Figure 12

Framework Method Steps

Note. Applying the Framework Method to Educational Standards. Adapted from “Using the Framework

Method for the Analysis of Qualitative Dyadic Data in Health Research,” by N. Collaco, R. Wagland, O.

Alexis, A. Gavin, A. Glaser, & E. K. Watson, 2021, Qualitative Health Research, 31(8), p. 1557

(https://doi.org/10.1177/10497323211011599) CC BY 4.0 DEED.

Step 7, interpret and report, of the framework method, as detailed in Figure 11, was used

by the coder in the review process. Step 8, reflect and recommend are used later in the chapter.

Findings

As it relates to the overarching question, "In what way(s) does the language found on the

2020 New Jersey student learning standards for social studies in Grades K–8 compare to Hess'

Cognitive Rigor Matrix definition of higher-order thinking?" The researcher found evidence of

depth and complexity in the standards from kindergarten through eighth grade. The standards

frequently call for analysis, evaluation, and synthesis, as shown in the frequency tables. Terms

such as evaluate, analyze, compare and contrast, and use evidence, which are hallmarks of

higher-order thinking, are seen throughout the standards. A distribution of depth of knowledge

(DOK) levels is shown in Figure 13 for Grades K–8 standards. The chart revealed:

• DOK 1 (recall and reproduction) comprised 3.4% of the standards.

• DOK 2 (skill/concept) comprised 37.6% of the standards.

• DOK 3 (strategic thinking and reasoning) comprised 40.6.0% of the standards.

• DOK 4 (extended thinking) comprises 18.2% of the standards.

In Hess' Cognitive Rigor Matrix, higher-order thinking is characterized by complex cognitive

processes, which are represented by the higher DOK levels:

• DOK 3 specifically involves strategic thinking, complex reasoning, and the ability to plan

and use evidence. This level requires students to go beyond text or rote application of

skills. At this level, students must be able to explain, support, or justify their thinking.

• DOK 4 requires extended thinking, complex analysis, and the ability to synthesize

information from multiple sources or contexts. Students at this level engage in tasks that

involve long-term investigation, research projects, and problem-solving. At this level

students spend more than one class period learning and applying information and creating

authentic connection to the material they are learning.

The language of the New Jersey standards, as indicated by the pie chart, aligns well with Hess'

definition of higher-order thinking. The chart evidences that there is a substantial emphasis on

DOK 3 and DOK 4 levels:

• The combined percentage for DOK 3 and DOK 4 is 58.8%, indicating that a majority of

the standards aim to engage students in strategic and extended thinking activities. This

suggests that the standards are designed to foster an educational environment where

students are encouraged to engage in higher-order thinking.

• The time that students are expected to dedicate to DOK 1 shows a minimal focus on

simple recall and reproduction, which are considered lower-order thinking skills.

• DOK 2 standards reflect a total of 37.6% of the standards. This indicates that

understanding the foundational skills and concepts is necessary, but they are not the sole

focus of the educational process.

Figure 13 demonstrates an alignment with Hess' Cognitive Rigor Matrix as it shows that

the standards place a strong emphasis on higher-order thinking skills, which are crucial for

preparing students not just for academic success but also for real-world problem-solving and

decision-making in an interconnected world, where global thinking is critical.

Figure 13

NJ Student Learning Standards in Grades K-8 by DOK Level

Note. How the questions are distributed based on Webb’s Depth of Knowledge between grades

K-8. Own Work.

Question 2 Findings

As it relates to question two, “How cognitively complex are the 2020 New Jersey student

learning standards for social studies in Grades K–8 as defined by Hess' Cognitive Rigor Matrix

The 2020 New Jersey student learning standards for social studies for Grades K–8, as analyzed

through Hess' Cognitive Rigor Matrix, demonstrated a significant level of cognitive complexity.

Hess' Cognitive Rigor Matrix combined Bloom's taxonomy with Webb's depth of knowledge

(DOK) to provide a more comprehensive view of cognitive demand in educational standards and

tasks. The cognitive complexity of the 2020 New Jersey student learning standards for social

studies in Grades K–8 is aligned with the higher levels of Hess' Cognitive Rigor Matrix because

the standards show a strong emphasis on strategic and extended thinking. This reflects an

educational approach that values critical thinking and analysis, as well as application and

synthesis and research and inquiry. The expected skills and learning, when taught appropriately

in schools, can prepare students not only to understand historical, civic, economic, and

geographic concepts but also to apply this understanding critically and creatively in various

contexts. Globally competent students can draw on and combine the disciplinary knowledge and

modes of thinking acquired in schools to ask questions, analyze data and arguments, explain

phenomena, and develop a position concerning a local, global, or cultural issue. Global thinking

routines capture key forms of thinking embodied in global competence. They are open-ended,

assuming no right or wrong answers. They are simple in design, which makes them useful for

teachers of varied levels of expertise yet cognitively sophisticated, providing extensive room for

growth and refinement. When teachers make these routines habitual practices—part of the way

we do things here—they pave the way for the kind of learning we need to prepare our youth for

our interdependence in the world (Mansilla, 2016).

Figure 14 indicates that the standards place an emphasis on foundational skills DOK 1

and DOK 2. These skills are crucial for building initial knowledge and understanding. However,

it should be noted that there is a considerable emphasis, 37%, on strategic thinking, which is

Webb’s depth of knowledge Level 3, indicating that even at this early stage, students are being

introduced to more complex cognitive tasks that require reasoning, applying knowledge, and

making connections. While extended thinking, Webb’s depth of knowledge in Level 4 tasks is

minimal. It allows educators to begin engaging students in multi-step problem-solving and

projects that require the integration of knowledge from various sources.

Figure 14

NJ Student Learning Standards in Grades K–2 by DOK Level

Note. How the questions are distributed based on Webb’s Depth of Knowledge between grades

K-2. Own Work.

For Grades 3–5, as shown in Figure 14, the emphasis on basic skills and concepts,

Webb’s depth of knowledge Level 2 is slightly higher, which is reflective of the need to solidify

foundational knowledge as students move into more complex subject matter. However, the

standards maintain a strong focus, 35%, on strategic thinking Webb’s depth of knowledge Level

3, which continues to foster students' abilities to analyze, compare, and infer. The increase in

extended thinking, Webb’s depth of knowledge Level 4, indicates that there is a progression

towards more demanding tasks that require extended reasoning and problem-solving.

Figure 15

NJ Student Learning Standards in Grades 3–5 by DOK Level

Note. How the questions are distributed based on Webb’s Depth of Knowledge between grades

3-5. Own Work.

Figure 16

NJ Student Learning Standards in Grades 6–8 by DOK Level

Note. How the questions are distributed based on Webb’s Depth of Knowledge between grades

6-8. Own Work.

As the middle school standards, Grades 6–8, are analyzed, standards make a substantial

shift towards higher-order thinking skills. The majority of the standards fall into Webb’s depth of

knowledge Level 3 and Level 4. These tasks require more strategic and extended thinking tasks.

These account for 85.3% of the standards. These standards will require a curriculum that is

heavily weighted towards tasks that demand deep engagement, such as conducting research,

engaging in complex reasoning, problem-solving, and synthesizing information across multiple

domains. There is minimal emphasis on Webb’s depth of knowledge in Level 2. There is also the

absence of Webb’s depth of knowledge Level 1, which is important.

This implies that with the standards, educators are expected to create learning

opportunities for students who move beyond simple recall. It requires educators to teach in a

manner that allows students to apply their foundational knowledge in more sophisticated ways.

When reviewing the New Jersey student learning standards for social studies in Grades

K–8, it is evident that there is a progression toward increasing cognitive complexity. The

standards in the early grades lay the groundwork with a strong base in foundational knowledge

and skills, yet they do not shy away from introducing students to tasks that require higher-order

thinking. As students progress, the focus shifts significantly towards strategic and extended

thinking, aligning with Hess’ CRM's definition of cognitive complexity. By the middle school

years, the standards predominantly involve tasks that engage students in complex, cognitively

demanding activities, preparing them for the challenges of higher education and the workforce.

This progression reflects a deliberate curricular design to develop critical thinking and problem-

solving skills as students advance through their educational journey.

Question 3 Findings

Question 3 looked at “How frequently do the 2020 New Jersey student learning standards

for social studies in Grades K–8 include higher-order thinking opportunities as identified in Hess'

Cognitive Rigor Matrix?” Throughout the Grade K–8 standards, students are encouraged to

apply knowledge across various contexts, not just within the area of social studies. There are

instances where students are asked to engage with contemporary issues, use evidence, and

support arguments, and they are also tasked with understanding the impact of historical events on

current society.

This cross-disciplinary application is aligned with Hess' Cognitive Rigor Matrix for

higher DOK levels. As evidenced in Figure 16, only 3% of the standards are at DOK level 1,

which is the lowest level of cognitive complexity in Webb’s depth of knowledge framework. At

least 58% of all standards in Grades K–8 are DOK level 3 or DOK level 4. DOK Level 3

standards require strategic thinking, complex and abstract thinking, and can have multiple

answers and approaches. This means the standards created if used in classrooms, can create

opportunities for students to develop critical thinking, problem-solving, and decision-making

skills, which have been shown to lead to global thinking. DOK Level 4 standards require

students to extend their thinking and focus on complex reasoning and planning. These standards

often require extended periods of time with multiple steps (Webb, 2002).

Figure 17

NJ Student Learning Standards in Grades K-8 by DOK Level

Note. How the questions are distributed based on Webb’s Depth of Knowledge between grades

K-8. Own Work.

Error Finding:

It should be noted that when the researcher conducted the AI, ChatGPT analysis for the

NJ student learning standards for social studies in Grades 3-5, an error message was present

which was noted in Appendix A. The error is noted in Figure 17.

Figure 18

AI Error Message

Note. Screenshot of an interaction showing an error message for analyzing the New Jersey social

studies standards in Grades 3–5, using Hess' Cognitive Rigor Matrix and Webb's depth of

knowledge levels. Own Work.

The AI was asked to explain the error message for guidance and it responded as follows:

“The message in the screenshot provided indicates that the user has reached the current usage

cap for GPT-4 and, therefore, cannot continue using GPT-4 until a specified time when the cap

resets.” Thus, it offered two alternatives:

Use the default model: This allows the user to continue interacting with the AI but with

the default model provided by OpenAI, which may not be GPT-4. The default model might be an

earlier version, like GPT-3 or GPT-3.5, which could potentially offer a different performance in

terms of understanding and generating responses.

Try again after the specified time: This suggests that the user can wait until the indicated

time when the usage cap for GPT-4 is reset. At this time, the user will be able to use GPT-4

again.

The usage cap is likely in place to manage demand and ensure fair access to the system

for all users. When you encounter such a message, it affects your query by potentially limiting

the complexity of the responses you can receive if you opt to use the default model instead of

GPT-4. The depth, accuracy, and nuances of the answers could be impacted depending on the

capabilities of the alternative model being used (ChatGPT, 2024).

The researcher was unaware if this may have caused coding errors. Because the coder

was not familiar enough with the multiple versions of ChatGPT the researcher was unable to

make a determination as if the coding process was impacted. The coding confirmation process

has been detailed in Appendix B with inconsistencies documented.

Coding Confirmation

Due to the nature of this study, the researcher chose to combine the framework method

with the Mayring step method. This allowed for leveraging the structural benefits of the

framework method, which is used for data organization and thematic analysis. The Mayring step

method allowed for detailed textual analysis. This approach is particularly useful in mixed-

methods research, where both broad thematic insights and detailed text interpretations are based

on a predetermined coding scheme (Mayring, 2000). During the coding process, the research

found disagreement with ChatGPT’s DOK-level assignment. Full details are shared in Appendix

The analysis of the New Jersey student learning standards for social studies using

artificial intelligence (AI), specifically ChatGPT, revealed some intriguing insights into what it

was capable of doing as well as what some of its limitations are when it comes to using it in

educational research. This is specifically when using it for extraction and coding of large

datasets. The researcher’s process involved running the standards through ChatGPT twice to

assess how the AI coded the standards based on Webb's depth of knowledge levels. The findings

highlighted a lack of consistency in the AI's coding process. Therefore, it was important to

investigate or at least better understand the implications of using AI for extracting and coding

large datasets.

Inconsistencies in AI Coding

While using the AI, ChatGPT, the researcher noted inconsistencies between the two

coding sessions. The coding of the dataset was performed at the grade level. This means the

researcher inputted the Grades K–2 standards, then Grades 3–5, and finally Grades 6–8. Each

grade level set was inputted in a unique session. The prompts were the same for all three grade

levels. The researcher did not perform all the coding sessions on the same day or on the same

computer. In fact, they were not even performed at the same time of day. This is important to

note because there are different versions of ChatGPT, and at peak times, an older version of the

AI may be used. When the researcher used ChatGPT to code the New Jersey student learning

standards for social studies, inconsistencies were noted. These can be attributed to several factors

in addition to those already mentioned. First, AI algorithms, including those used by ChatGPT,

learn from vast amounts of data to make inferences. However, these inferences can vary due to

the nuances in the data it was trained on and the inherent ambiguity in natural language. For

example, in Webb’s depth of knowledge, Level 2 and Level 3 both list “compare” as a verb in

their respective levels. The difference between when it is level 2 versus level 3 is when the

students are asked to apply the skill; it is a level 2 verb. When using reasoning to compare, it

becomes a level 3 verb. Additionally, educational standards, like the New Jersey student learning

standards for social studies, often contain complex, multifaceted concepts that can be interpreted

differently, leading to differences in how AI categorizes these standards into Webb's depth of

knowledge levels.

Concerns with Using AI for Large Dataset Coding

The inconsistencies observed raised important concerns regarding the reliability of using

AI, like ChatGPT, for coding large datasets. The variability the researcher found in coding this

dataset shows the potential impact that relying solely on AI can have on the quality of research

findings.

These inconsistencies can lead to potentially flawed interpretations or conclusions.

Relying on AI to process and code large datasets, the inconsistencies should be a concern to

other researchers, and they should be cautious. It emphasizes the need for human oversight and

verification. Due to the limitations presented by AI, the role of the researcher as a subject matter

expert becomes incredibly important. For this study, the researcher manually coded the New

Jersey student learning standards for social studies.

The researcher leveraged their expertise in the subject matter and their familiarity in

using Webb's depth of knowledge framework. This manual approach allowed for more targeted

interpretations of the standards. By leveraging AI in educational research through coding

standards, both its potential and limitations were revealed. Even though AI offers promising

opportunities for analyzing large datasets, human expertise and oversight are crucial. By

incorporating subject matter experts into the research process, researchers help manage AI's

limitations.

Table 1

Frequency Table Grades K-2

Note. Comparison of depth of knowledge (DOK) level totals identified by ChatGPT and

researchers for Grades K–2. Own Work.

The researcher continued to find some disagreement with ChatGPT coding in Grades 3–

5. In reviewing the categorization of the depth of knowledge (DOK) levels between the ChatGPT

responses and those determined by the researcher, minor discrepancies were noted across the

categories. For instance, ChatGPT identified 58 items at DOK 2 (skill/concept), while the

researcher identified 56, as shown in Table 2. This shows that there is some marginal variation in

the categorization process. Similarly, a slight difference was observed at DOK 3 (strategic

thinking and reasoning), with ChatGPT identifying 37 items and the researcher 35. It was

important to acknowledge these differences. It should be noted that they are not deemed

substantial enough to warrant a more detailed statistical analysis at this moment. Overall, the

researcher found that the trends in categorization were consistent between the two sources.

Table 2

Frequency Table Grades 3–5

Note. Comparison of depth of knowledge (DOK) level totals identified by ChatGPT and

researchers for Grades 3–5. Own Work.

The bigger discrepancies between ChatGPT and the researcher appear in Grades 6–8. As

seen in Figure 11, the researcher deemed nine more standards to be classified as DOK Level 2

than ChatGPT, which categorized them as DOK Level 3. Eight of these nine standards

specifically used “explain how” and “explain why,” which Webb’s depth of knowledge

specifically categorizes as DOK Level 2 (Webb, 2002). Hence, the researcher categorized them

as DOK Level 2, as noted in Table 3.

Table 3

Frequency Table Grades 6-8

Note. Comparison of depth of knowledge (DOK) level totals identified by ChatGPT and

researchers for Grades 6–8. Own Work.

Findings Summary

On June 3, 2020, the New Jersey Board of Education adopted new social studies

standards that were to inform education for all public school students in Grades K–12.

The social studies standards were created to prepare students because “Today’s challenges are

complex, have global implications, and are connected to people, places, and events of the past.

The study of social studies focuses on a deep understanding of concepts that enable students to

think critically and systematically about local, regional, national, and global issues. Authentic

learning experiences that enable students to apply content knowledge, develop social studies

skills, and collaborate with students from around the world prepare New Jersey students for

college, careers, and civic life” (New Jersey Department of Education, 2020, Para 3).

This purposeful shift meant that the New Jersey Department of Education recognized that the

world was becoming more interconnected and that students in New Jersey needed to prepare for

a more interconnected and interdependent future.

The 2020 intent, spirit, and mission for the New Jersey social studies standards align with

the United Nations sustainable development goals, specifically targeting Goal 4: Education. The

SDGs emphasize the importance of inclusive and quality education as a means to empower

individuals and promote social justice.

The purpose of this mixed-methods study was to analyze the New Jersey student learning

standards for social studies in grades K-8 using Hess' Cognitive Rigor Matrix. Standards were

coded first using artificial intelligence, ChatGPT, and then reviewed by the researcher using the

framework method. The standards were coded using Webb’s depth of knowledge Level 1

through Level 4. The assessment of these standards as performed by AI, ChatGPT, and the

researcher showed that these standards if executed in the classroom from kindergarten through

eighth grade, may prepare students to think critically and be active and positive contributors to

local, regional, national, and global issues. The standards not only focus on introducing students

to higher-order thinking and complex cognitive skills and ideas but also progressively increase

the DOK levels from kindergarten to eighth grade. For example, from kindergarten to second

grade, there are approximately five DOK level 1, standards 27 DOK Level 2, 20 DOK level 3,

and only two DOK level 4. This means that the New Jersey social studies standards in Grades K–

8 align them with the UN's sustainable development goals, focusing on quality education. Using

AI and cognitive analysis tools, it was found that these standards progressively built critical

thinking and problem-solving skills, preparing students to address local and global issues

effectively.

Conclusion

It was evident through analysis conducted in this study that the New Jersey social studies

standards for Grades K–8 are setting the stage for New Jersey students to have a bright future.

When exposed to these standards in the classroom, students are not just about memorizing dates

and facts. They are being tasked with thinking bigger and deeper, not just about historical figures

or events but about modern-day issues. This affords students the opportunity to develop the

global thinking skills that the state of New Jersey so purposely emphasized in its mission and

vision when it developed these standards. From the minute students arrive in kindergarten, they

are ushered to the standards set goals. When delivered proficiently by teachers and mastered by

students, the standards can set the learners on a path where learning is more than just taking

notes. With these standards, social studies classes have the opportunity to be places that are

about asking questions, analyzing different sides of a story, and finding solutions. When

delivered properly, they afford students the opportunity to see the past, the present, and even the

future through different lenses. This approach is key to getting them ready for a world that is

dynamic on a daily basis.

When looking toward the future, it is pretty exciting to see that students will have the

opportunity to grow up equipped with the kind of thinking that can tackle global challenges

head-on. These standards are lofty in that they require a new way of thinking and teaching. The

road to teaching these standards may be bumpy with all the learning gaps, but the education

community is still smoothing out from the pandemic. However, they can become a benchmark

for quality high-level instruction to help students navigate through life. As students learn and

master these standards, they can work together to create ideas that can make a real difference,

understand other people from around the world, and learn how to work together. So, for the

learner sitting in classrooms today and for the society waiting for them out there, this is more

than just education; it is preparation for a world that needs their smart, compassionate, and

leadership. The standards are not just about shaping scholars but global citizens ready to step up

and create a better future for everyone.

This research journey was embarked on to explore the presence of cognitive complexity

and global thinking in the 2020 New Jersey student learning standards for social studies in

Grades K–8. The researcher used the lens of Hess' Cognitive Rigor Matrix and Webb's depth of

knowledge levels to assess higher-order thinking in these. The findings support that when the

standards were created, there was a deliberate curricular emphasis on higher-order thinking

skills. If the standards are executed as intended by well-trained teachers, and these skills are

mastered by students, they are better prepared for the demands of the modern world, where

global and critical thinking capabilities are necessary.

During this research, an intriguing and unexpected subplot arose. Inconsistencies were

found between the AI and the researcher as well as the AI and itself, when it was asked to assess

and categorize the standards using the exact same prompts. This variability raised consequential

reliability concerns in educational research contexts.

These inconsistencies are not just technical glitches. They indicate a broader issue

inherent when using artificial intelligence programs to code educational content. The nuanced

discrepancies observed, which could be attributable to factors such as the time of coding, the

device used, or even the version of AI engaged with, highlighted the complex, dynamic nature of

AI algorithms and their interpretations of educational standards.

While AI is new and offers unlimited potential, the researcher cautions against over-

relying on it for the critical task of coding large datasets in educational research. The

implications for future research are profound. AI offers unprecedented scalability and efficiency

in handling large datasets; however, its current limitations necessitate a more balanced

approach.

Chapter 5: Summary, Conclusions, and Recommendations

Chapter 5 provides a summary of the findings, as well as conclusions and

recommendations for policymakers and any potential future researchers. In order to do so, it is

important to understand that education continues to change at an alarming rate. The world

changes so fast that one no longer knows what kind of future schools are preparing students for.

At the time this research was being compiled, education was changing rapidly. This can be

attributed to the national teacher shortage, which can be blamed on teacher burnout, high

demands from parents, lack of social-emotional support, and even lack of transition from the

COVID-19 pandemic, in which many teachers and students had not addressed the trauma they

faced. Technological resources are being marketed to districts and to educators daily. Many

promise to make huge gains yet do not have the field data to support their promises. As teachers

burn out and opportunity gaps widen, school leaders are more amenable to possible solutions that

can stop the bleeding.

In addition to what is happening in education, the world is dealing with advancements in

technology that is threatening to disrupt democracies around the world. The war that was going

on in Europe between Russia and Ukraine threatened to destabilize the European Union and its

allies. Additionally, the Israeli-Palestinian conflict was far from being resolved and possibly in

one of the most hostile phases anyone has ever seen. For once, the United States and its

international allies did not see eye-to-eye. The election of 2024 in the United States promises to

be one of the most important elections in American and world history.Where does that leave

society?

In order for education to be effective and to support students in their success in an ever-

changing world, it must prepare them with the skills needed in the 21st century which will enable

them to understand themselves as part of an interconnected world, where they can develop the

collaborative skills and critical thinking skills they need to succeed.

While this seems simple enough, one must take into account that teachers and schools are

still dealing with learning loss that was a result of the global pandemic. For this reason, many

may ask how educators can begin to focus on global thinking and critical thinking when many

students still are not able to read or do mathematics at their grade level. While this is a valid

concern, it should be noted that critical thinking and experiential learning opportunities that give

students agency over their learning and foster global thinking are not in addition to, but a

different approach to thinking about how students are taught while simultaneously preparing

them for the future.

Purpose

This study set out to answer three research questions. The first question was, “In what

ways does the language found in the 2020 New Jersey student learning standards for social

studies in Grades K–8 compare to Hess' Cognitive Rigor Matrix definition of higher-order

thinking?” It was evident from the onset, as the researcher began to assess the standards from

kindergarten all the way through eighth grade, that there was a purposeful and meaningful

intention to include specific verbiage that aligned with Hess' Cognitive Rigor Matrix as well as

Webb's depth of knowledge. For Grades K–2, the standards incorporate a balanced mix of

Webb's depth of knowledge levels, with the majority falling under level 2 (skill/concept). Both

the AI and researcher's coding results were quite consistent, suggesting that students were

introduced to both basic recall and some more sophisticated skills at an early age.

As students progress from Grades 3–5, there was a noticeable shift towards higher DOK levels,

with the bulk of the standards focusing on DOK 2 (skill/concept) and DOK 3 (strategic thinking

& reasoning). The AI and researcher totals for DOK 4 (extended thinking) differed, indicating

some room for interpretation at the most complex level of thinking; however, this shift was

nominal, and none significantly altered the experience for the students in these grades.

In Grades 6–8, a significant increase in skills related to DOK 3 and DOK 4 levels was seen. The

researcher identified more opportunities for extended thinking than AI. This shift in increasing

levels of Webb’s depth of knowledge as students get older suggests that as students advance, the

curriculum increasingly challenges them with complex and abstract thinking tasks. The skills

they are exposed to and expected to develop are more rigorous and challenging than expected.

It is important to understand how the findings relate to the research questions. The first question

asked, “In what way(s) does the language found on the 2020 New Jersey student learning

standards for social studies in Grades K–8 compare to Hess' Cognitive Rigor Matrix definition

of higher-order thinking?”

When the language in the New Jersey social studies standards was compared to Hess'

Cognitive Rigor Matrix, it was evident that there was a progression in cognitive complexity.

This is evident in the verbiage and skill requirements as students progress through Grades K–8.

The language and demands reflect a transition from basic recall to advanced analytical and

evaluative tasks. For children, this progression means that as they move from Grades K–8, they

are expected to develop and sharpen their thinking skills. As they enter kindergarten, they spend

some of their social studies time on basic recall, like remembering facts and details. However, it

should be noted that this is less than 10% of the time, as noted in the findings in Chapter 3. As

learners get older, they are encouraged to think more deeply. They are tasked with analyzing

information and evaluating different viewpoints, as well as citing evidence and using different

resources. This kind of learning prepares them to handle more complex problems and think

critically, to be successful in today's world. When education systems emphasize higher-order

thinking from an early age, students are able to develop problem-solving and critical thinking

skills, which are important as they get older and become contributing members of a global

society. Children who grow up with problem-solving and critical thinking skills can question the

status quo, innovate, and find better ways of doing things. With these skills, and in the future,

these students will have the ability to be more engaged as citizens, ready to tackle big issues like

climate change, inequality, and global health crises.

The second question the research set out to answer was, “How cognitively complex are

the 2020 New Jersey student learning standards for social studies in Grades K–8 as defined by

Hess' Cognitive Rigor Matrix?”

Looking at how cognitive complexity is defined by Hess' Cognitive Rigor Matrix and

Webb’s depth of knowledge and distributed according to New Jersey standards, it is evident that

they are well-distributed. Standards appear to move upward as students move from Grades K–8.

This means that as children progress through their academic careers, they become more

challenged and are able to develop higher-order thinking skills. This indicates a deliberate effort

to develop higher-order thinking skills in students throughout their educational journey.

The final research questions asked, “How frequently do the 2020 New Jersey student

learning standards for social studies in Grades K–-8 include higher-order thinking opportunities

as identified in Hess' Cognitive Rigor Matrix?” The frequency of higher-order thinking

opportunities increases with each grade level. Grades K–2 are the foundational grades. Here, the

groundwork for learning, understanding, and making sense of social studies concepts and ideas is

set. In Grades 3–5, there are more intermediate skills developed, and by the time students reach

middle school, Grades 6–8, the standards provide more frequent and advanced higher-order

thinking opportunities. As students progress from Grades K– 8, they are afforded more chances

to flex their brain muscles. By the time students arrive in middle school, they are tackling the

kind of cognitive complexity that prepares them for the real world. New Jersey's social studies

standards for social studies in Grades K–8 provide students with a strong foundation that allows

them to develop the global thinking skills that will help them to be successful in a world, no

matter the challenges they face.

Discussions and Implications

In this study, Hess' Cognitive Rigor Matrix was used to assess higher-order thinking in

GradeK–8 social studies standards by using artificial intelligence (AI) as a coder. As AI has not

been widely applied, thus, it was important to explore its implications for educational policy on

its effectiveness in research, and the dynamic role it can play as a component of research.

Throughout this research, AI proved to be a powerful tool for dissecting understanding and the

cognitive complexity of these standards. In addition to its use as a coder, AI became a partner in

research. In some cases, AI played an unofficial mentor role in the research process. The

researcher was able to get recommendations and suggestions on the articles to read. It enabled

the creation of idea maps.

It helped the researcher better understand readings by providing summaries and

simplified explanations. An important finding in this research was how AI can be used to support

students in higher education. The use of artificial intelligence can help tailor educational content,

identify areas of difficulty, and suggest additional resources to help students succeed. This

unique dynamic significantly shaped the research process, offering new perspectives and

capabilities that enriched the analysis and understanding of the research process. This

personalized approach can help bridge educational gaps that students from disadvantaged

backgrounds might face or fill gaps that polylingual students can encounter due to meet code-

switching.

While AI was a wonderful tool for coding large datasets in a short amount of time, its

inconsistencies highlighted the indispensable value of human oversight in contextualizing and

reviewing its findings to ensure they aligned with research objectives. A human review was

essential not only for interpreting and validating the results but also for ensuring that the research

remained ethical. Through this symbiotic relationship, human intelligence and artificial

intelligence can work together to enhance understanding of complex educational phenomena.

The ability to learn continuously is an intriguing aspect of using AI in research. As AI

algorithms process more data, the accuracy and efficiency of their analyses should improve,

suggesting that AI can yield more refined insights over time. It is important to note that OpenAI's

ChatGPT does not learn and update its knowledge base in real time as a result of user

interactions. The model's learning process occurs during its training phase, where it is fed vast

amounts of text data. Despite this, due to its complex algorithms and the vast amount of training

data it has encountered, the AI model's interpretation can vary slightly between sessions. This

can lead to different emphases or nuances being picked up in each response, as was seen in this

study. This study may contribute to the conversation about the use of artificial intelligence in

education. It shows how AI can affect education research and policy, as well as the need to

balance AI's abilities with human skills. In the future, combining AI with education can greatly

enhance research and help students.

Suggestions for Future Research

In this study, global thinking was assessed in the New Jersey social studies standards in

Grades K–8. Based on this research, the standards create opportunities for students to develop

the global thinking skills, the New Jersey Department of Education set forth in its mission

statement and vision statement. There are three potential studies that would provide valuable

insight into their effectiveness. The first would be to engage in a barrier assessment. Researchers

are encouraged to create a study that can identify any barriers to effectively teaching these

standards in a manner that achieves the cognitive complexity learning opportunities intended by

the standards.

Future studies could look into resource limitations, teacher training gaps, or systemic

educational inequalities. Understanding these roadblocks is important because, without the right

tools and support, those standards might not reach their full potential.

Another opportunity to research is to take a closer look at how schools are actually putting these

standards into practice. Curriculum implementation is critical. Delving into how these standards

are being implemented across various schools and districts and what factors contribute to

successful or challenging integration would provide meaningful feedback. Quality standards are

important, and the New Jersey Department of Education has been successful in developing them.

However, teaching and learning are not just about handing teachers a new set of guidelines; it is

about seeing how they bring them to life in the classroom. Are some schools able to implement

these standards effectively while others struggle? It is vital to understand what is making the

difference. This is not just about making sure every student is being provided equitable access; it

is about sharing what is working so that every school can make the same gains.

It is also important to engage in longitudinal studies. Tracking how students are doing

over time can provide feedback for improving these standards or for developing future standards.

It is not enough to know they have the opportunity to be exposed to critical thinking and global

issues. It has to be documented if these skills are maintained and carried beyond the classroom.

Are they becoming the problem-solvers and world-shakers? By following their journey, it will be

apparent if what is being taught now is prepping them for the real world.

Finally, when it comes to AI, it is crucial to understand why there is a disagreement in

coding standards. Research could be conducted to assess whether allowing AI to ask more

guiding questions before prompting it to code the standards can enable it to better understand the

nuances of coding. The question arising is whether it would it be possible for AI to better code

standards if Bloom's taxonomy framework and Webb's depth of knowledge framework were

uploaded?

Considerations for Policymakers

We stand at the crossroads of history, where the decisions made today forge the path for

the generations of tomorrow. Education is our most potent weapon in the pursuit of freedom and

equality. With that said it must be accessible to every child, regardless of their zip code or the

circumstances of their birth. Today’s students hold the promise of our nation's future within their

capable hands. Each child deserves to walk into classrooms that serve not just as rooms with

whiteboards but as gateways to forge critical thinking skills and develop an understanding of

their role in an interconnected global village with many challenges to be faced.

We are in a very precarious time. Policymakers need to act very assertively and

decisively, yet as a nation, we stand more divided than ever before. The first call is for

policymakers to act with the courage that our times demand. For standards such as these to be

successful, policymakers must ensure every school is equipped with the resources needed to

thrive. They must ensure that educators are supported now more than ever, as we are already

dealing with mass educator attrition across the country. Quality educators have the ability to

nurture the minds of children who will one day solve the puzzles that are not being solved now.

Quality educators are the only ones who can break down the barriers of inequality so that

every young child has the opportunity to rise, unbounded by the ceilings that confined their

predecessors. When it comes to education, we should never be bound by what is politically

expedient. We should be propelled to act with the conviction of what is right. Investing in

education with unwavering belief is not merely an instrument of personal advancement but the

bedrock of democracy and progress. Policymakers must act with urgency now more than ever.

When providing children with the opportunity to think critically and logically and when giving

them agency over their learning, we lift them up. Agentic students are not just taught to dream

but to reason, not just to aspire, but to engage, not just to reach for the stars, but to navigate the

cosmos. When education is done purposefully, rigorously, and when global thinking is the norm,

schools do more than educate; they liberate. Policymakers’ charges may be complex, but they are

not complicated. Thus, they need to ensure:

• Resource allocation: Continue to ensure that schools have the necessary resources, such

as adequate funding, modern technology, and access to quality materials.

• Teacher support: Providing robust training programs, ongoing professional development,

and support systems for teachers can help them adapt to and effectively teach the new

standards. Assuring that teachers have the quality leadership they deserve at the school,

district, county, and state levels is vital.

• Equity in education: Continuing to address systemic inequalities that affect students'

ability to benefit from the standards is crucial. This includes considering the needs of

students from diverse socioeconomic, cultural, and language backgrounds.

• Assessment methods: Developing formative assessment tools that accurately measure the

cognitive complexity and higher-order thinking skills outlined in the standards to help

ensure that educational objectives are being met. These should be seamless tools that are

easily integrated into classroom instruction to keep things simple for teachers.

• Feedback loops: Establishing mechanisms to receive feedback from educators, students,

and the community about how the standards are working in practice can help

policymakers make informed adjustments.

• Sustainability: Policies should consider the sustainability of education reforms, including

how to maintain and build upon the changes brought about by the standards.

• Ethics: Students and educators should have guidelines for the responsible use of AI,

safeguarding their interests.

Students also viewed