Category: Higher Education

Categories
Higher Education Language Development

Comparing the effects of ChatGPT and automated writing evaluation on students’ writing and ideal L2 writing self

Using a randomized controlled experimental design, Shi et al. (2025) compared the effects of ChatGPT-based feedback and traditional automated writing evaluation (AWE) systems on English-as-a-foreign-language (EFL) students’ writing performance and their ideal L2 writing self. One hundred and fifty second-year university students from three writing classes in a Chinese public university were recruited and randomly divided into a ChatGPT group, an AWE group, and a control group.

After an eleven-week intervention, results showed that ChatGPT helped students perform better in their writing compared to the control group and the AWE group, but compared to the AWE group, ChatGPT significantly lowered students’ ideal L2 writing self. Qualitative results shed light on possible causes: while participants were fully aware of the affordances of ChatGPT feedback, they were also concerned with their (over) reliance on the tool and the accompanying loss of creativity and agency and expressed their reserved attitude toward future intention to use ChatGPT.

Educators should refine learning objectives based on students’ ZPD and design prompts accordingly, so that ChatGPT supports learning rather than completing tasks, while also teaching prompt-engineering skills. For lower-intermediate to intermediate learners, AWE’s systematic and rule-based feedback can provide stronger scaffolding and better preserve authorship. However, ChatGPT’s richer affordances may lead to over-reliance, weakening learner agency and diminishing the ideal L2 writing self. Therefore, language-education goals should be redefined to incorporate AI literacy and critical thinking, safeguarding teacher and learner agency and promoting responsible use.

 

Source (Open Access): Shi, H., Chai, C. S., Zhou, S., & Aubrey, S. (2025). Comparing the effects of ChatGPT and automated writing evaluation on students’ writing and ideal L2 writing self. Computer Assisted Language Learning, 1-28.

https://doi.org/10.1080/09588221.2025.2454541… Read the rest

Categories
Higher Education

The Role of Undergraduates’ Critical Thinking in Generative AI Reliance Behaviors

Hou and colleagues conducted a large-scale survey study using structural equation modeling (SEM) to examine how undergraduates’ critical thinking influences their different types of reliance on generative AI during problem-solving tasks. The study analyzed 808 valid responses, measuring students’ critical thinking skills and dispositions, AI literacy, trust in AI, and four types of AI-use behaviors—reflective, cautious, collaborative, and thoughtless use. The authors conceptualized reliance behavior as the way learners evaluate and make use of the differences between AI and human abilities, and proposed that critical thinking may play a key moderating role in this process.

The results showed that AI literacy strongly predicted both critical thinking skills (β = .66, p < .001) and dispositions (β = .41, p < .001), whereas trust in AI was negatively related to both (skills: β = –.16, p < .05; disposition: β = –.11, p < .001). Regarding reliance behaviors, critical thinking skills were positively associated with collaborative use (β = .25), reflective use (β = .21), and cautious use (β = .24), with similar effects found for critical thinking disposition. These findings highlight the importance of critical thinking in supporting desirable forms of AI use. In contrast, trust strongly predicted thoughtless use (β = .47, p < .001) and also slightly increased collaborative use (β = .15, p < .05) and reflective use (β = .19, p < .001), indicating a dual role of trust in both strengthening and weakening ideal reliance behaviors. More importantly, AI literacy promoted collaborative (β = .25), reflective (β = .20), and cautious use (β = .22) through the mediation of critical thinking, whereas trust produced negative indirect effects on these desirable behaviors because it reduced critical thinking (β = –.05 to –.06, p < .001). This means that critical thinking both enhances the positive influence of AI literacy and suppresses the potential blind reliance brought by high trust, guiding learners toward more reflective, careful, and collaborative ways of using AI.

Overall, the study provides strong evidence that critical thinking does not simply reduce AI reliance; instead, it shapes how students rely on AI, encouraging forms of use that are more reflective, collaborative, and prudent. The authors argue that the development of AI literacy must be accompanied by the cultivation of critical thinking to reduce thoughtless dependence and to promote healthier human–AI collaboration. They also emphasize that educational interventions should clearly define “ideal reliance behaviors” and help students develop responsible and thoughtful habits of AI use in an era where generative AI is becoming increasingly widespread.

Source (Open Access): Hou, C., Zhu, G., & Sudarshan, V. (2025). The role of critical thinking on undergraduates’ reliance behaviours on generative AI in problem‐solving. British Journal of Educational Technology, 56(5), 1919-1941.

https://doi.org/10.1111/bjet.13613… Read the rest

Categories
Higher Education Social and Motivational Outcomes

Learners’ Preferences for Feedback from AI and Human Instructors

Le and his team examined whether learners’ preferences for feedback from human instructors versus generative artificial intelligence (AI) would change after receiving feedback from different sources and interface types in an academic English writing task. The study recruited 114 university students who were non-native English speakers and randomly assigned them to four groups: no feedback (control), human instructor feedback, ChatGPT 4.0 in a free-conversation interface, and a structured writing analysis tool powered by ChatGPT. Learners’ preferences were measured both before and after the task using rating scales and binary-choice questions, and the four groups were compared in terms of post-task preference and preference change.

The results showed that learners already had a clear preference for human instructors before the task (87.2% chose human), and this preference remained stable after the task (86.0% chose human), reflecting a phenomenon of algorithm aversion in educational settings. However, post-test preference scores differed significantly among the four groups: the human instructor group rated significantly higher than both the free-conversation AI group and the control group. On the binary human/AI choice measure, significant differences were also found — the human instructor and structured AI tool groups both scored higher than the free-conversation AI group. Regarding preference change, the overall mean shift was close to zero, but the differences among groups were significant: the free-conversation AI group showed a slight increase in preference for AI, whereas the human instructor and structured AI tool groups remained more favorable toward humans. In other words, although all three feedback types were effective, the free-conversation interface was the only one that reduced algorithm aversion and increased learners’ acceptance of AI, while the structured, one-time feedback tool further reinforced their preference for human instructors.

Based on these findings, the authors argue that enhancing the interactivity and dialogic nature of AI-based learning tools may influence learners’ preferences more effectively than purely improving their technical performance. Interactive dialogue allows for clarification and correction, which reduces learners’ unrealistic expectations that algorithms must be perfect and mitigates distrust. Overall, the study situates human preference within the context of interface design, providing both empirical insights and cautions for the adoption, product design, and pedagogical integration of AI in education.

 

Source (Open Access): Le, H., Shen, Y., Li, Z., Xia, M., Tang, L., Li, X., … & Fan, Y. (2025). Breaking human dominance: Investigating learners’ preferences for learning feedback from generative AI and human tutors. British Journal of Educational Technology.

https://doi.org/10.1111/bjet.13614… Read the rest

Categories
Achievement Higher Education

How International Roommates Shape Academic Success in Low-Income Colleges

Tsai and Trinidad (2025) investigate how intercultural roommate pairings affect college outcomes among low-income U.S. students. The study centers on Berea College, a tuition-free liberal arts institution in Kentucky that primarily serves economically disadvantaged populations. Leveraging institutional data from over 6,600 domestic students between 2000 and 2015, the authors evaluate whether being paired with an international roommate in the first year influences academic performance and persistence throughout college.

Using quasi-experimental methods, including inverse probability weighting (IPW) and robustness checks through augmented and matching estimators, the researchers compare students with and without international roommates while accounting for demographic factors such as race, gender, and transfer student status. The findings reveal that domestic students paired with international roommates achieved significantly higher first- and second-year GPAs (approximately 0.14 and 0.10 points higher, respectively) and showed a modest improvement in second-year retention (about four percentage points). However, the benefits gradually diminished over later years, and no significant effects were found for long-term persistence or six-year graduation rates.

The authors interpret these results through the lens of peer effects and diversity in higher education. They suggest that intentional intercultural roommate pairings create structured opportunities for cross-cultural engagement that may counteract homophily in predominantly white, low-income settings. This exposure not only enhances academic habits and motivation but also fosters inclusivity and openness among domestic students.

Overall, the study provides empirical evidence that low-cost, policy-driven diversity interventions, such as pairing domestic and international students, can meaningfully improve early academic outcomes for disadvantaged students. While the effects do not extend to graduation, the research highlights the importance of designing inclusive residential environments that promote sustained intercultural interaction as a pathway toward educational equity.

 

Source (Open Access): Tsai, H. T. A., & Trinidad, J. E. (2025). Effect of International Roommates on College Outcomes: Evidence from Students of Disadvantaged Backgrounds. Educational Policy, 08959048251315481.

https://doi.org/10.1177/08959048251315481… Read the rest

Categories
Educational Administration and Leadership Higher Education

Predicting program-specific first-year persistence in higher education using a person–environment fit perspective

A recent predictive modeling study by de Vries et al. (2025) examined how different conceptualizations of person–environment (PE) fit predict program-specific persistence among first-year undergraduates. Data were collected from 1305 first-year  students across five non-selective bachelor’s programs at a large research-intensive university (Biomedical Sciences, Business Administration, Health & Life Sciences, Law, and Movement Sciences), with at least 200 students per programme.

Using logistic regression with LASSO regularization and 10-fold cross-validation, the authors compared five clusters of predictors: interest fit, ability fit, prior achievement/ personality/ motivation, choice process, and background characteristics. Both profile correlations and polynomial regression approaches were applied to operationalize interest fit. High school exam grades and self-reported abilities captured ability fit.

Results showed that models predicted persistence with moderate accuracy: 67–77% in training samples and 50–75% in testing samples. Models performed better in correctly classifying persisters than non-persisters. Interest fit was the most consistent predictor, with polynomial regression-based measures retained in four of the five disciplinary models (small-to-moderate effect sizes). Profile correlation indices also contributed in some programs, but less strongly. Ability fit through subject-specific high school grades (e.g., biology for Biomedical Sciences, mathematics for Business Administration) emerged as another robust predictor. In contrast, traditional indicators such as HSGPA, personality traits, and motivational measures had limited additional value once interest and subject-specific grades were included. Choice process variables (e.g., depth of exploration) and background characteristics showed only marginal contributions.

The study highlights the importance of discipline-specific interests and ability matches in predicting first-year persistence. While the models are more effective at forecasting persistence than dropout, the findings stress the need for program-specific approaches to student success and suggest that admissions and advising practices should focus more on interest and subject fit rather than broad indicators like GPA or personality.

 

Source (Open Access): de Vries, N., Merkle, B., Meeter, M., Janke, S., Bakker, T. C., & Huizinga, M. (2025). Predicting program-specific first-year persistence in higher education using a person-environment fit perspective. European Journal of Higher Education, 1-22.

https://doi.org/10.1080/21568235.2025.2502536… Read the rest

Categories
Higher Education Programme Evaluation

Leap, learn, earn: exploring academic risk taking and learning success across gender and socioeconomic groups

A recent cross-sectional study by Hübner and Pfost (2024) investigates gender-related and SES-related disparities in academic risk-taking (ART) and their implications for learning success in undergraduate education. Drawing on a sample of 381 German university students, the study addresses four research questions: (1) Are there significant differences in ART levels based on students’ social group affiliation (gender, SES)? (2) Does ART predict students’ learning success? (3) Are there significant indirect effects of social group affiliation on learning success via ART (mediation)? (4) Are there significant differences in the strength of the relationship between ART and learning success depending on students’ social group affiliation (moderation)?

The authors employed structural equation modeling to address these questions. Specifically, ART was assessed across two dimensions—seminar group and peer group—while learning success was measured through students’ subjective evaluations of their own learning achievement in the current seminar. Gender was coded as female, male, or diverse, with the latter category excluded due to its small sample size. SES was dichotomized into high and low groups. Higher education entry qualification (HEEQ) was included as a general control variable, based on students’ self-reported prior academic performance (ranging from 1 = very good to 6 = insufficient).

The findings reveal significant gender differences in ART on both the seminar group dimension (Mmale = 3.53, Mfemale = 3.23, F = 11.83, p = 0.001, d = 0.40) and the peer group dimension (Mmale = 3.27, Mfemale = 3.49, F = 5.58, p = 0.018, d = 0.28). In contrast, no significant differences emerged between high-SES students and low-SES students in either the seminar group dimension (Mlow-SES = 3.26, Mhigh-SES = 3.33, F = 0.74, p = 0.390, d = 0.09) or the peer group dimension (Mlow-SES = 3.39, Mhigh-SES = 3.45, F = 0.51, p = 0.476, d = 0.07). Both the seminar group (β = 0.23, p = 0.004) and peer group (β = 0.21, p = 0.009) dimensions of ART significantly predicted learning success. Regarding mediation effects, the indirect path from gender through the seminar group dimension of ART to learning success was significant (β = -0.04, p = 0.015), whereas the peer group dimension did not function as a mediator. Furthermore, gender (β = 0.10, p = 0.004) and SES (β = 0.10, p = 0.018) significantly moderated the relationship between the peer group dimension of ART and learning success, but no moderation effects were observed for the seminar group dimension.

This study is among the first to confirm gender differences in ART within higher education and to demonstrate the beneficial effects of ART on learning success using inferential statistical methods. By addressing the four research questions, it contributes to a more nuanced understanding of ART and its role in the reproduction of educational inequalities.

 

Source (Open Access): Hübner, V., & Pfost, M. (2024). Leap, learn, earn: exploring academic risk taking and learning success across gender and socioeconomic groups. Higher Education, 1-19.

 https://doi.org/10.1007/s10734-024-01307-w… Read the rest