International Electronic Journal of Elementary Education

Editorial for the Special Issue on Large Scale Assessment: Challenges and Innovations

Ummugul Bezirhan; Murat Doğan Şahin — Sun, 23 Mar 2025 00:00:00 +0300

This editorial introduces the IEJEE’s Special Issue on Large Scale Assessment: Challenges and Innovations, highlighting emerging themes and methodological advancements in educational measurement. The selected studies focus on process data utilization to examine test-taker behavior, innovations in psychometric modeling for assessment, classification, and the influence of social-emotional learning on academic achievement. This editorial discusses the contributions of the included studies, their implications for future research, and the evolving role of AI, machine learning, and digital assessment technologies in shaping the future of large-scale assessments.

Decoding Student Insights: Analyzing Response Change in NAEP Mathematics Constructed Response Items

Congning Ni, Bhashithe Abeysinghe, Juanita Hicks — Sun, 23 Mar 2025 00:00:00 +0300

The National Assessment of Educational Progress (NAEP), often referred to as The Nation’s Report Card, offers a window into the state of U.S. K-12 education system. Since 2017, NAEP has transitioned to digital assessments, opening new research opportunities that were previously impossible. Process data tracks students’ interactions with the assessment and helps researchers explore students’ decision-making processes. Response change is a behavior that can be observed and analyzed with the help of process data. Typically, response change research focuses on multiple-choice items as response changes for those items is easily evident in process data. However, response change behavior, while well known, has not been analyzed in constructed response items to our knowledge. With this study we present a framework to conduct such analyses by presenting a dimensional schema to detect what kind of response changes students conduct and how they are related to student performance by integrating an automated scoring mechanism. Results show that students make changes to grammar, structure, and the meaning of their response. Results also revealed that while most students maintained their initial score across attempts, among those whose score did change, factor changes were more likely to improve scores compared to grammar or structure changes. Implications of this study show how we can combine automated item scoring with dimensional response changes to investigate how response change patterns may impact student performance.

Running Out of Time: Leveraging Process Data to Identify Students Who May Benefit from Extended Time

Burhan Ogut, Ruhan Circi, Huade Huo, Juanita Hicks, Michelle Yin — Sun, 23 Mar 2025 00:00:00 +0300

This study explored the effectiveness of extended time (ET) accommodations in the 2017 NAEP Grade 8 Mathematics assessment to enhance educational equity. Analyzing NAEP process data through an XGBoost model, we examined if early interactions with assessment items could predict students’ likelihood of requiring ET by identifying those who received a timeout message. The findings revealed that 72% of students with disabilities (SWDs) granted ET did not use it fully, while about 24% of students lacking ET were still actively engaged when timed out, indicating a considerable unmet need for ET. The model demonstrated high accuracy and recall in predicting the necessity for ET based on early test behaviors, with minimal influence from background variables such as eligibility for free lunch, English Language Learner (ELL) status, and disability status. These results underscore the potential of utilizing early assessment behaviors as reliable predictors for ET needs, advocating for the integration of predictive models into digital testing systems. Such an approach could enable real-time analysis and adjustments, thereby promoting a fairer assessment process where all students have the opportunity to fully demonstrate their knowledge.

Investigating the Differential Relationship Between the Big Five Domains of Social and Emotional Skills and Mathematics Achievement

Mihriban Altiner Sert, Serkan Arıkan — Sun, 23 Mar 2025 00:00:00 +0300

The current study explores the differential relationship between social and emotional learning (SEL), based on the Big Five personality traits, and mathematics achievement among Turkish high school students. Using data from the OECD’s 2019 Survey on Social and Emotional Skills (SSES), it examines how SEL dimensions predict math outcomes and how these relationships vary by gender, socioeconomic status (SES), and level of SEL evaluation in schools. Key findings reveal that open-mindedness and emotional regulation positively correlate with math achievement, while high social engagement shows a negative association. Girls' SEL skills had a stronger predictive value for math achievement than boys', and SEL had a more substantial impact on students from lower SES backgrounds. Formal SEL assessments in schools was also related to higher math scores. These results emphasize the importance of SEL programs tailored to specific demographic needs, particularly for disadvantaged students, and suggest that formal SEL assessment in schools could enhance academic outcomes.

Improving Context Scale Interpretation Using Latent Class Analysis for Cut Scores

Liqun Yin, Ummugul Bezirhan, Matthias Von Davier — Sun, 23 Mar 2025 00:00:00 +0300

This paper introduces an approach that uses latent class analysis to identify cut scores (LCA-CS) and categorize respondents based on context scales derived from large-scale assessments like PIRLS, TIMSS, and NAEP. Context scales use Likert scale items to measure latent constructs of interest and classify respondents into meaningful ordered categories based on their response data. Unlike conventional methods reliant on human judgments to define cut points based on item content, model-based approaches such as LCA find statistically optimal groups, a categorical latent variable, that explains item score differences based on score distribution differences between latent classes. Cut scores for these classes are determined by conditional probability calculations that relate class membership to observed scores, finding the intersection point of adjacent smoothed probability distributions and connecting it to the construct. Demonstrated through application to PIRLS 2021 data, this is useful to validate existing categorizations of the context scale by human experts, and can also help to enhance classification accuracy, particularly for scales exhibiting highly skewed distributions across diverse countries. Recommendations for researchers to adopt this LCA-CS approach are provided, demonstrating its efficiency and objectivity compared to judgment-based methods.

Latent Profile Analysis: Comparison of Achievement versus Ability-Derived Subgroups of Mathematical Skills

Onur Demirkaya, Sharon Frey, Sid Sharairi, JongPil Kim — Sun, 23 Mar 2025 00:00:00 +0300

This study compared latent profiles derived from student subgroups of varying levels of mathematical skills defined by achievement and ability assessment scores. Achievement and ability cut scores for identifying students at both ends of the mathematics spectrum were applied and the resulting latent profiles within each condition were compared. The research utilized latent profile analysis to identify student profiles with achievement scores from the Iowa Assessments and ability scores from CogAT. The participants consisted of 50,998 second-grade students in a Southeastern state. The finding revealed varying demographics and patterns of ability and achievement for each condition, underscoring the need to acknowledge students with diverse learning styles and the distinct dynamics between achievement and ability scores to use for identifying students who may benefit from tailored educational programs.

Exploring Test Taking Disengagement in the Context of PISA 2022: Evidence from Process Data

Başak Erdem Kara — Sun, 23 Mar 2025 00:00:00 +0300

Achievement tests are commonly used in education to evaluate students' academic performance and proficiency in specific subject areas. However, there is a major problem that threatens the validity of achievement test scores which is test-taking disengagement. Respondents provide answers that are inconsistent with their true ability level and can introduce construct irrelevant variance that threatens the validity of scores. This study examines test-taking disengagement in the context of PISA 2022 using process data to identify patterns of behavior that influence student performance. Three key indicators; response time, number of actions and self-reported effort, were used to examine engagement levels. Employing Latent Profile Analysis (LPA), distinct profiles of test-takers were identified, ranging from highly engaged to disengaged groups. Results indicate that disengagement, characterized by low self-reported effort, minimal interactions, and rapid responses, is associated with lower test performance, threatening the validity of scores. These findings highlight the significance of accounting for disengagement when interpreting the results of large-scale assessments. The implications were discussed in relation to the existing literature and recommendations for future research were provided to address identified gaps and extend the study's contributions.