Korean Language Education Research

Sign In
한 EN
ISSN (P) 1225-8571 ISSN (E) 2713-699X

Article Info

How Teacher-educators and ChatGPT Evaluate Pre-service Teachers’ Writing? — Focused on Newspaper Editorials

Ahn Sanghee Kim Kyounghwa Kim Youngran

Jeonbuk National University 청주대학교 Kangwon National University

Korean Language Education Research Vol. 60 No. 5 pp.5-39 (2025)

DOI: 10.20880/kler.2025.60.5.5

Abstract

This study explores how teacher educators (TEs) and ChatGPT con- struct and apply evaluation criteria in assessing pre-service teachers’ (PTs) writing, focusing on university newspaper editorials. Nineteen PTs majoring in Korean language education participated in the study. Three TEs collaboratively developed evaluation criteria and a rubric to rate the texts, while ChatGPT (Plus version) was instructed to create its own ru- bric and later assess the same texts using the TEs’ rubric. Both TEs and ChatGPT emphasized logical reasoning and persua- siveness as core elements of the editorial genre. However, TEs developed detailed, content-oriented criteria highlighting argumentative depth and reader awareness, whereas ChatGPT presented integrated, form-oriented criteria emphasizing clarity, structure, and media adaptability. When ap- plying the same rubric, TEs exhibited a wider scoring range with greater qualitative sensitivity, while ChatGPT showed a convergent mid-range scoring pattern. These results indicate that ChatGPT can apply teacher-developed rubrics consistently but lacks contextual and interpretive nuance. The scoring results of ChatGPT also showed low discriminatory power of the scores. It may serve as a supplementary tool that helps teachers refine criteria & rubric design or cross-check their practice of evaluation, sug- gesting potential for AI–human collaboration in writing evaluation.

Keywords

AI-assisted writing evaluationteacher educatorsgenerative AIChatGPTrubric development

References

  1. [1] [학술지(정기간행물)] AlAfnan M. A./Chatgpt as an educational tool: Opportunities, challenges, and recommendations for communication, business writing, and composition courses/Journal of Artificial Intelligence and Technology/2023/3(2)/60~68//
  2. [2] [학술대회논문] Allam H./Artificial intelligence in education: an argument of Chat-GPT use in education/2023 9th International Conference on Information Technology Trends (ITT)/2023//151~156/IEEE
  3. [3] [학술지(정기간행물)] Awidi, I. T./Comparing expert tutor evaluation of reflective essays with marking by generative artificial intelligence (AI) tool/Computers and Education: Artificial Intelligence/2024/6/100226~//
  4. [4] [학술지(정기간행물)] Cha J. W./A Comparative Study of AI-Based Writing Assessment Tools and Human Evaluators - Focusing on book report texts -/Korean Language and Literature in International Context/2024/(102)/681~705//
  5. [5] [학술지(정기간행물)] Choi J. Y./Calibrating human and machine: Developing an AI-assisted rater discussion model for automated writing evaluation(AWE)/Journal of Korean Association for Educational Information and Media/2025/31(4)/1297~1329//
  6. [6] [학술지(정기간행물)] Choi, S. G./Development and Validation of Quality Assessment Tools for AI-Powered Automated Writing Feedback/Journal of CheongRam Korean Language Education/2025/103/227~263//
  7. [7] [학술지(정기간행물)] Herbold S./A large-scale comparison of human-written versus ChatGPT-generated essays/Scientific reports/2023/13(1)/18617~//
  8. [8] [학술지(정기간행물)] Imran M./Analyzing the role of ChatGPT as a writing assistant at higher education level: A systematic review of the literature/Contemporary Educational Technology/2023/15(4)/ep464~//
  9. [9] [학술지(정기간행물)] Jang, S. M./ChatGPT has Changed the Future of Writing Education - Focusing on the response of writing education in the era of artificial intelligence -/Research on Writing/2023/56/7~34//
  10. [10] [학술지(정기간행물)] Jauhiainen, J. S./Generative AI in education: ChatGPT-4 in evaluating students’ written responses/Innovations in Education and Teaching International/2025/62(4)/1377~1394//
  11. [11] [학술지(정기간행물)] Kim, J. Y./The Effects and Limitations of Using AI-Based Automatic Writing Feedback Programs in Writing Classes - Focusing on the Case of Using KEEwi in Reading & Expression Classes -/The Journal of Yeongju Language & Literature/2024/58/355~386//
  12. [12] [학술지(정기간행물)] Kolade, O./Is AI changing learning and assessment as we know it? Evidence from a ChatGPT experiment and a conceptual framework/Heliyon/2024/10(4)/~//
  13. [13] [학술지(정기간행물)] Lee, Y. B./A Comparative Study on the Characteristics and Validity of Instructor, Peer, and AI Feedback in College Writing Education : Focusing on Common Issues in Students’ Argumentative Texts/Korean Journal of General Education/2025/19(3)/19~33//
  14. [14] [학술지(정기간행물)] Lee, Y. B./Reframing College Writing Education in the Era of Generative AI － An Integrated ‘AI-Assisted Collaborative Writing Pedagogy’ Combining Process-Based and Genre-Based Instruction with AI Literacy/The Korean Journal of Literacy Research/2025/16(4)/175~207//
  15. [15] [학술지(정기간행물)] Mao, J./Generative artificial intelligence in education and its implications for assessment/TechTrends/2024/68(1)/58~66//
  16. [16] [학술지(정기간행물)] Megawati R./The role of GPT chat in writing scientific articles: A systematic literature review/Jurnal Penelitian Pendidikan IPA/2023/9(11)/1078~1084//
  17. [17] [학술지(정기간행물)] Park S. Y./Exploring the Potential of AI Tools in University Writing Assessment: Comparing Evaluation Criteria between Humans and Generative AI/Journal of practical engineering education/2024/16(5)/663~676//
  18. [18] [학술지(정기간행물)] Rudolph J./ChatGPT: Bullshit spewer or the end of traditional assessments in higher education?/Journal of applied learning and teaching/2023/6(1)/342~363//
  19. [19] [학술대회논문] Shidiq, M./The use of artificial intelligence-based chat-gpt and its challenges for the world of education; from the viewpoint of the development of creative writing skills/Proceeding of international conference on education, society and humanity/2023/1(1)/353~357/
  20. [20] [학술대회논문] Smolansky, A./Educator and student perspectives on the impact of generative AI on assessments in higher education/Proceedings of the tenth ACM conference on Learning@ Scale/2023//378~382/
  21. [21] [학술지(정기간행물)] Son, H. J./Perceptions and attitudes of English learners towards the utilization of AI-based Automated Writing Evaluation (AWE) programs and instructor feedback/English Language & Literature Teaching/2023/29(3)/91~112//
  22. [22] [학술지(정기간행물)] Son, H. S./Studying the Potential Use of Generative AI in College Writing/The Journal of General Education/2025/(31)/41~72//
  23. [23] [학술지(정기간행물)] Steiss, J./Comparing the quality of human and ChatGPT feedback of students’ writing/Learning and Instruction/2024/91/101894~//
  24. [24] [학술지(정기간행물)] /////~//
  25. [25] [학술지(정기간행물)] Wang, J./Evaluating rater accuracy and perception for integrated writing assessments using a mixed-methods approach/Assessing Writing/2017/33/36~47//
  26. [26] [학술지(정기간행물)] Yan D/Detection of AI-generated essays in writing assessments/Psychological Test and Assessment Modeling/2023/65(1)/125~144//

Busan National University, 2nd College of Education Building, Room 315 (Prof. Eun Jung Jeon’s Office), Jangjeon-dong, Geumjeong-gu, Busan, Republic of Korea Tel: +82-51-510-2602 Society E-mail: koredu1991@daum.net Editorial Board E-mail: koredu1991_edit@daum.net Copyright © 2026 Association of Korean Language Education Research. All Rights Reserved.