Effects of Generative AI–Based Writing Feedback and Learner Responses
KWON, TAE HYEON
충북대학교
Korean Language Education Research Vol. 61 No. 1 pp.37-66 (2026)
Abstract
This study compared the effects of feedback generated by human evaluators and ChatGPT on high-school students’ revision performance in argumentative writing to investigate the educational effectiveness and characteristics of generative AI-based writing feedback. A quasi-experimental pretest-posttest design was used with 154 second-year high school students to examine quantitative effects and identify qualitative features through a learner perception survey. The results showed that the effect size of AI feedback was slightly larger than that of human feedback, and the two-way ANOVA revealed a significant main effect only for writing proficiency level. According to qualitative analysis, AI feedback demonstrated strengths in specificity, feasibility, systematicity, consistency, and a focus on improving linguistic expression, but it also had limitations related to stereotypical patterns and insufficient support for metacognitive engagement. These results suggest that generative AI feedback can be used effectively as a tool for initial revision, whereas a hybrid approach that combines it with teacher feedback is required to address higher-order argumentative elements and promote metacognitive processes.
Keywords
생성형 AI쓰기 피드백고쳐쓰기인간 피드백챗지피티
References
- 권태현(2024), 「ChatGPT를 활용한 쓰기 채점 및 피드백 방안–프롬프트 전략을 중심으로-」, 『새국어교육 』 141, 7 - 42.
- 박영민·가은아·권태현·박종임·박찬흥·이수진·이재기(2025), 『작문교육론』. 서울: 역락.
- 심지연(2024), 「생성형 AI 를 활용한 한국어 쓰기 피드백 방안-ChatGPT 와 한국어 교사의 쓰 기 피드백에 대한 학습자 인식 및 선호도 차이를 중심으로」, 『Journal of Korean Culture (JKC)』 67, 45 - 79.
- 최숙기(2024), 「국어 교사의 챗지피티 (ChatGPT)를 활용한 작문 피드백 경험에 대한 사례 연 구」, 『교원교육』 40(1), 379 - 404.
- 최진영·김형성·송보라·김지수(2023), 「생성형 인공지능 ChatGPT를 활용한 고등학생의 작문 과제 고쳐쓰기 양상」, 『리터러시 연구』 14(6), 79 - 119.
- Applebee, A. N. & Langer, J. A. (2011), “EJ Extra: A snapshot of writing instruction in middle schools and high schools [free access]”. English Journal 100(6), 14 - 27.
- Bandura, A. (1997), Self - efficacy: The exercise of control (Vol. 11). New York: Freeman.
- Dai, W., Tsai, Y. S., Lin, J., Aldino, A., Jin, H., Li, T., ... & Chen, G. (2024), “Assessing the proficiency of large language models in automatic feedback generation: An evaluation study”, Computers and Education: Artificial Intelligence 7, 100299.
- Escalante, J., Pack, A., & Barrett, A. (2023), “AI - generated feedback on writing: Insights into efficacy and ENL student preference”, International Journal of Educational Technology in Higher Education 20(1), 57.
- Evans, C. (2013), “Making sense of assessment feedback in higher education”. Review of educational research, 83(1), 70 - 120.
- Graham, S., Harris, K., & Hebert, M. (2011), Informing Writing: The Benefits of Formative Assessment. A Report from Carnegie Corporation of New York, New York: Carnegie Corporation of New York.
- Guo, K., Pan, M., Li, Y., & Lai, C. (2024), “Effects of an AI - supported approach to peer feedback on university EFL students’ feedback quality and writing ability”, The Internet and Higher Education 63, 100962.
- Jacobsen, L. J. & Weber, K. E. (2025), “The promises and pitfalls of large language models as feedback providers: A study of prompt engineering and the quality of AI - driven feedback”, AI 6(2), 35.
- Jacobsen, L. J., Mertens, U., Jansen, T., & Weber, K. E. (2025). AI, Expert or Peer?- - Examining the Impact of Perceived Feedback Source on Pre - Service Teachers Feedback Perception and Uptake. arXiv preprint arXiv:2507.16013.
- Lin, S. & Crosthwaite, P. (2024), “The grass is not always greener: Teacher vs. GPT - assisted written corrective feedback”, System 127, 103529.
- Lo, L. S. (2023), “The CLEAR path: A framework for enhancing information literacy through prompt engineering”, The Journal of Academic Librarianship 49(4), 102720.
- Mekheimer, M. (2025), “Generative AI - assisted feedback and EFL writing: a study on proficiency, revision frequency and writing quality”, Discover Education 4(1), 170.
- Mizumoto, A. & Eguchi, M. (2023), “Exploring the potential of using an AI language model for automated essay scoring”, Research Methods in Applied Linguistics 2(2), 100050.
- Ranalli, J. (2021), “L2 student engagement with automated feedback on writing: Potential for learning and issues of trust”, Journal of Second Language Writing 52, 100816.
- Rashkin, H., Clark, E., Huot, F., & Lapata, M. (2025), “Help Me Write a Story: Evaluating LLMs’ Ability to Generate Writing Feedback”, In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 25827 - 25847.
- Stahl, M., Biermann, L., Nehring, A., & Wachsmuth, H. (2024), “Exploring LLM prompting strategies for joint essay scoring and feedback generation”, arXiv preprint arXiv:2404.15845.
- Steiss, J., Tate, T., Graham, S., Cruz, J., Hebert, M., Wang, J., ... & Olson, C. B. (2024), “Comparing the quality of human and ChatGPT feedback of students’ writing”, Learning and Instruction 91, 101894.
- Wan, T. & Chen, Z. (2024), “Exploring generative AI assisted feedback writing for students’ written responses to a physics conceptual question with prompt engineering and few - shot learning”, Physical Review Physics Education Research 20(1), 010152.
- Yoon, S. Y., Miszoglad, E., & Pierce, L. R. (2023), “Evaluation of ChatGPT feedback on ELL writers’ coherence and cohesion”, arXiv preprint arXiv:2310.06505.
- Zhan, Y. & Yan, Z. (2025), “Students’ engagement with ChatGPT feedback: Implications for student feedback literacy in the context of generative artificial intelligence”, Assessment & Evaluation in Higher Education, 1 - 14.
- Zhang, Z., Aubrey, S., Huang, X., & Chiu, T. K. (2025), “The role of generative AI and hybrid feedback in improving L2 writing skills: a comparative study”, Innovation in Language Learning and Teaching, 1 - 19. Zimmerman, B. J. &. Schunk, D. H. (2011), “Self - regulated learning and performance: An introduction and an overview” In B. Zimmerman & H. Dale(Eds.), Handbook of self - regulation of learning and performance, Florence, KY: Routledge.