Education EDU530 Week 5 assignment

Running head: DIGITAL TOOL IN EDUCATION AND FORMATIVE ASSESSMENT 2

DIGITAL TOOLS IN EDUCATION AND FORMATIVE ASSESSMENT 2

Digital Tools in Education and Formative Assessment

St Thomas University

EDU 530 Educational Measurement

Dr. Pamela Dahlin

January 26, 2025

Digital Tools in Education and Formative Assessment

Step 1: Topic Exploration

Chosen Topic: Digital Tools in Education and Formative Assessment

Research Summary: Digital tools included in education and formative assessment are the go-to in education. The three findings from the peer-reviewed research disclose the following:

1. Challenges in Teachers' Adoption: Borte et al. (2023) highlight that incongruence in the realm of technology versus pedagogy prevents teachers from taking advantage of the full potential inherent in digital tools-stimulating student-centered learning or even formative assessment practices definition and more teacher training in data literacy.

2. Evidence of Effectiveness: According to Sortwell et al. (2024), formative assessment practices positively influence the learning process for students in K-12 if done appropriately. However, more research is needed, given the low certainty of evidence concerning optimal strategies.

3. Personalized Learning: Weigand, Trgalova, and Tabach (2024) have discussed how digital tools, especially adaptive systems, allow individualized learning experiences and formative assessments in mathematics education. These tools provide real-time feedback that helps students and teachers address specific learning needs.

4. Gaps in Current Research : All three note that digital formative assessment strategies' effectiveness and long-term impact on students' engagement and outcomes remain under-researched.

5. Challenges for Teachers: Some challenges that hold back teachers are the limitations of the technology itself, the lack of training to use these technologies, and the lack of research on best practices for using these technologies within formative assessment.

Step 2: Narrow Focus

Narrowed Topic:

The potential of Artificial Intelligence (AI) ventures to realize an impact on enhancing formative assessment through personalized feedback and specific challenges in teachers' implementation of ventures.

Relevance to Assessment and Measurement:

This topic is relevant because formative assessment is a cornerstone of effective teaching and learning. Other challenges include equity, teacher autonomy, and professional development when integrating AI. It can lead to insightful ways to design practical AI-driven formative assessment tools and further knowledge of the broader implications for K-12 education.

Step 3: Crafting the Guiding Question

Guiding Research Question:

How could AI-embedded formative assessment improve personal feedback and decision-making in K-12 education, and what does that indicate for the workload and burden of teachers?

Rationale:

The guiding question cuts into what is critical and timely: technology and education must work at a crossroads that transform AI possibilities within formative assessment. Therefore, it is because AI provides unlimited ways of constructing productive personalized feedback that can eventually be useful to teachers and educators in providing teaching strategies that can accommodate needs that will be different for different learners; it promises improved results in better learning. For example, it would not be easy to get data education for those teachers by using new technologies and weighing those tools with professional judgment. However, there is an entire issue of equity, access, and ethical issues regarding the use of AI in many K-12 settings. In keeping with the ongoing debate about new inventions in measurement and assessment, there seems to be some search for possibilities in harmony with restrictions about the prospect and usage of formative assessments via AI. For these reasons, such research allows for actual development in ways that will help make any final equitable, vigorous AI-supported assessments feed into teachers and eventually improve the quality of teaching and learning for a regiment of students across all contexts. Additionally, frameworks like the Learning-with-PA Model highlight the importance of contextual factors and digital agents in mediating learning outcomes (Dolata et al., 2023).

References

Børte, K., Lillejord, S., Chan, J., Lillehaug, B. W., & Greiff, S. (2023). Prerequisites for teachers’ technology use in formative assessment practices: A systematic review.  100568 ; Educational Research Review ; 41. https://doi.org/10.1016/j.edurev.2023.100568

Dolata, M., Katsiuba, D., Wellnhammer, N., & Schwabe, G. (2023). Learning with Digital Agents: An Analysis based on the Activity Theory.  Journal of Management Information Systems,  40(1), 56–95. https://doi.org/10.1080/07421222.2023.2172775

Sortwell, A., Trimble, K., Ferraz, R., Geelan, D. R., Hine, G., Ramirez-Campillo, R., Carter-Thuiller, B., Gkintoni, E., & Xuan, Q. (2024). A Systematic Review of Meta-Analyses on the Impact of Formative Assessment on K-12 Students’ Learning: Toward Sustainable Quality Education.  Sustainability,  16(17), NA. https://doi.org/10.3390/su16177826

Weigand, H.-G., Trgalova, J., & Tabach, M. (2024). Mathematics teaching, learning, and assessment in the digital age. ZDM, 56. https://doi.org/10.1007/s11858-024-01612-9

Running head: THE BASED OF PREDICTIONS & RATIONALE 2

THE BASE OF PREDICTIONS & RATIONAL 2

The Base of Predictions & Rationale

St Thomas University

EDU 530 Educational Measurement

Dr. Pamela Dahlin

February 2, 2025

The Base of Predictions & Rational

The growing use of digital technologies in education informs my understanding of AI's potential in formative assessment. AI is used in educational technology to deliver real-time feedback, tailor learning, and help instructors. According to my coursework and readings, formative evaluation allows instructors to detect knowledge gaps, measure progress, and alter teaching (Sortwell et al., 2024). However, AI's ability to improve formative assessment faces severe practical difficulties. Many instructors have challenges using AI-driven formative assessment systems because of insufficient training, equity issues, and technical limits (Børte et al., 2023). AI can create individualized feedback on a scale, but its quality and trustworthiness compared to human review are disputed.

One of my primary assumptions is that AI-embedded formative assessment will enhance K-12 individualized feedback and decision-making. Previous research has shown that AI can assess student replies, discover trends, and customize feedback to specific learning requirements (Weigand et al., 2024). AI-driven assessment systems might save instructors from grading and assessing student work, letting them concentrate on teaching and student engagement. However, AI may depersonalize education and reduce instructors' participation in formative evaluation. I assume that AI models are bias-free, yet research reveals that many AI systems inherit biases from their training data, which might lead to inequitable evaluation results (Dolata et al., 2023). Identifying bias in AI models raises questions about how AI-generated feedback can affect diverse student groups, especially underprivileged ones.

The prediction is that AI-embedded formative assessment systems will reduce teacher effort as they improve. Formative assessment is time-consuming. Therefore, AI's capacity to automate processes like grading multiple-choice questions, delivering quick feedback, and creating progress reports should help instructors save time. However, instructors must still analyze AI-generated insights and ensure that feedback meets students' requirements. Higher-order thinking abilities like critical analysis and creativity are frequently best assessed via open-ended replies and classroom discussions. AI may not be able to do so. While AI has shown potential in essay scoring and natural language processing, its capacity to deliver nuanced feedback on complicated student work needs additional study (Sortwell et al., 2024). AI will complement teacher-driven formative evaluation, not replace it.

One of my uncertainties regarding AI-embedded formative evaluation is my long-term impact on student engagement and motivation. While real-time feedback may help students improve, AI-driven assessment may reduce intrinsic motivation. Impersonal or mechanistic AI comments may deter students. AI-driven formative assessment systems may favor digitally savvy youngsters, creating a digital divide between students with and without home devices. Research suggests equitable access and teacher support enhance digital formative assessment (Borte et al., 2023). These worries suggest further research into how AI might boost student engagement and digital equity.

My academic interests include education, technology, and assessment techniques. So, I am inspired by AI in assessment, and as someone interested in new teaching and learning approaches, personalized training and assessment using AI-driven technologies intrigues me. Teachers and students need to know how AI can improve formative assessment, which impacts student learning. I would like to understand how AI can help teachers with their decision-making while using technologies for education. When AI is integrated into education, ethics, and practical challenges arise: data privacy, risk of algorithmic bias, human assessment supervision, and others must be addressed. The research question I stated previously is “How could AI-embedded formative assessment improve personal feedback and decision-making in K-12 education, and what does that indicate for the workload and burden of teachers?” The research question has a transparent meaning for K-12 education and policy. Formative assessment with AI might revolutionize schools with scalable, tailored learning. However, educators and technology developers should collaborate to solve AI implementation challenges and make these tools effective and equitable. Also, this research may contribute to education technology discussions by examining how AI-driven formative assessment influences teacher and student learning. This work may inspire professional development programs to help educators employ AI-based assessment systems. As teachers' training delays technology uptake in education (Borte et al., 2023), practical professional development approaches are essential for this research.

I aim to raise awareness of how AI may enhance formative assessment for equity, teacher burden, and student engagement. I want to answer my research question to produce ethical and practical AI-driven formative assessments. However, this work may shape future schooling to ensure AI supports teachers. Education must critically assess its effects to guarantee that AI breakthroughs match educational best practices and student learning objectives.

References

Borte, K., Lillejord, S., Erduran, S., Wasson, B., & Greiff, S. (2023). Prerequisites for teachers’ technology use in formative assessment practices: A systematic review. Educational Research Review, 100568–100568. https://doi.org/10.1016/j.edurev.2023.100568

Dolata, M., Katsiuba, D., Wellnhammer, N., & Schwabe, G. (2023). Learning with Digital Agents: An Analysis based on the Activity Theory. Journal of Management Information Systems, 40(1), 56–95. https://doi.org/10.1080/07421222.2023.2172775

Sortwell, A., Trimble, K., Ferraz, R., Geelan, D. R., Hine, G., Ramirez-Campillo, R., Bastian Carter-Thuiller, E. Gkintoni, & Xuan, Q. (2024). A systematic review of meta-analyses on the impact of formative assessment on K-12 students’ learning: Toward sustainable quality education. Sustainability, 16(17), 7826–7826. https://doi.org/10.3390/su16177826

Weigand, H.-G., Trgalova, J., & Tabach, M. (2024). Mathematics teaching, learning, and assessment in the digital age. ZDM. https://doi.org/10.1007/s11858-024-01612-9

Running head: THE SEARCH & ANNOTATED BIBLIOGRAPHY 2

THE SEARCH & ANNOTATED BIBLIOGRAPHY 2

The Search & Annotated Bibliography

St Thomas University

EDU 530 Educational Measurement

Dr. Pamela Dahlin

February 9, 2025 

The Search & Annotated Bibliography

K-12 AI-embedded formative assessment may alter tailored learning and teaching. AI in formative examinations helps teachers provide more personalized feedback, saving time on grading. AI can identify knowledge gaps, track student progress, and personalize learning. However, its execution causes teacher load, student engagement, and equity issues. AI can automate and scale feedback, but it may depersonalize education and reduce instructors' formative assessment duties and feedback quality. Due to biases, AI systems may judge various student groups differently. This study critically analyses academic perspectives to examine the merits and downsides of AI-embedded formative assessment in the classroom. Fairness, teacher support and training, and AI technologies that enhance human participation in education will be prioritized. Understanding these interactions is essential for ethical K-12 formative assessment using AI.

Borte, K., Lillejord, S., Erduran, S., Wasson, B., & Greiff, S. (2023). Prerequisites for teachers’ technology use in formative assessment practices: A systematic review. Educational Research Review, 100568–100568. https://doi.org/10.1016/j.edurev.2023.100568

Borte et al. (2023) provide a systematic review of formative assessment technology needs for teachers. The report uses 22 studies to examine teachers' adoption of digital tools. Evaluation topics include clear definitions of formative assessment and congruence with the instructional approach. This foundation helps teachers use technology without losing focus on educational goals. According to the review, instructors require data literacy to analyze and apply digital tool results to enhance student learning. The authors recommend teacher training, help, and stronger conceptual frameworks for formative assessment technology.

Methodological rigor and peer-reviewed journal credibility improve systematic review findings. The authors' educational research, technology integration, and assessment competence validate their conclusions. Twenty-two studies fully cover instructor issues. Teaching training, more explicit frameworks, and more significant support systems are research contributions. The assessment may not provide educators with actionable recommendations. The study highlights crucial issues but does not provide an easy answer for teachers.

Since it emphasizes the need to employ technology in assessment procedures, the relevance of this study to AI-embedded formative assessment is notable. Digital technology integration and data literacy training for teachers influence AI systems in formative assessment. This article emphasizes teacher readiness and strong conceptual frameworks to help K-12 education adopt AI technology. Unlike previous articles, the study focuses on AI applications rather than theory or technology. This tendency toward practical insights makes it a valuable resource for educators negotiating the challenges of introducing AI into assessment systems, especially as AI becomes more advanced and needs careful preparation.

Dolata, M., Katsiuba, D., Wellnhammer, N., & Schwabe, G. (2023). Learning with Digital Agents: An Analysis based on the Activity Theory. Journal of Management Information Systems, 40(1), 56–95. https://doi.org/10.1080/07421222.2023.2172775

Dolata et al. (2023) examine educational digital agents that might speak natural language to students. Digital agents' effects on learning outcomes and student-tool interactions are investigated using activity theory. Technology in the classroom affects students' behavior, according to activity theory. The study emphasizes digital agents' potential to provide customized, interactive learning experiences and proposes a paradigm for their use in education. The study illuminates how digital agents affect learning and how AI may enhance formative assessment.

The authors, specialists in digital learning technologies and educational institutions, employ activity theory to support their conclusions. The study's methodologies and theoretical framework explain digital agents' involvement in education, establishing the platform for future research. Research on digital agents' influence on K-12 education is lacking. The theoretical investigation is intense, but real-world evidence on how these tools improve student results or teaching practices would deepen the case and illuminate the 12 formative assessments' practical consequences. The relevance of this study to AI-driven formative assessment is crucial because it explains how AI-powered digital agents might engage with students to deliver tailored learning and feedback. AI tool exploration overlaps with increased interest in how technology may improve formative assessment by personalizing feedback to specific students. Formative assessment aims to promote continuous, tailored learning, and the research provides a theoretical framework for AI's potential in personalized learning. Dolata et al. (2023) emphasize AI tool-learner interaction, unlike other sources that focus on practical issues and instructor preparation for AI integration. This new perspective on AI's educational function sheds light on how digital agents may change formative assessment and feedback.

Sortwell, A., Trimble, K., Ferraz, R., Geelan, D. R., Hine, G., Ramirez-Campillo, R., Bastian Carter-Thuiller, E. Gkintoni, & Xuan, Q. (2024). A systematic review of meta-analyses on the impact of formative assessment on K-12 students’ learning: Toward sustainable quality education. Sustainability, 16(17), 7826–7826. https://doi.org/10.3390/su16177826

Sortwell et al. (2024) study meta-analyses to determine how formative assessment affects K-12 learning. According to the research, formative assessment improves student learning, although its impact varies by the type of assessment used. Feedback-based assessments perform better than formative tests, which may have variable outcomes. The analysis also shows that previous studies lack methodological rigor, highlighting the need for more substantial, long-term research on formative assessment, particularly when technology is included. The review contributes to understanding how formative assessment approaches might enhance educational results in K -12 settings by combining information from several meta-analyses. The authors assess many papers to ensure high-quality evidence, yet several meta-analyses in the study have low confidence. This suggests that the evaluation is based on a large body of research, but the evidence is inconsistent, and some findings may be unreliable. The work is published in Sustainability, a peer-reviewed education sustainability publication, ensuring academic credibility. However, AI-enhanced formative assessment techniques are underexplored. The evaluation focuses on conventional formative assessment approaches and their potential to improve learning outcomes, not AI systems. While Sortwell et al. (2024) do not concentrate on AI in formative assessment, their results provide a vital foundation for K-12 education. The review finding that formative assessment improves learning opens the door to potential advantages of AI-driven formative assessments. The research also contributes by examining formative assessment's efficiency, which is vital for assessing how AI technologies may be incorporated into such processes. This source emphasizes formative assessment's relevance in enhancing learning outcomes and highlights research gaps that might guide AI's application in formative assessment systems.

References

Borte, K., Lillejord, S., Erduran, S., Wasson, B., & Greiff, S. (2023). Prerequisites for teachers’ technology use in formative assessment practices: A systematic review. Educational Research Review, 100568–100568. https://doi.org/10.1016/j.edurev.2023.100568

Dolata, M., Katsiuba, D., Wellnhammer, N., & Schwabe, G. (2023). Learning with Digital Agents: An Analysis based on the Activity Theory. Journal of Management Information Systems, 40(1), 56–95. https://doi.org/10.1080/07421222.2023.2172775

Sortwell, A., Trimble, K., Ferraz, R., Geelan, D. R., Hine, G., Ramirez-Campillo, R., Bastian Carter-Thuiller, E. Gkintoni, & Xuan, Q. (2024). A systematic review of meta-analyses on the impact of formative assessment on K-12 students’ learning: Toward sustainable quality education. Sustainability, 16(17), 7826–7826. https://doi.org/10.3390/su16177826