Effects of AI Authorship Information on Writing Assessment — Changes Across Text Quality Levels and Analytic Scoring Criteria
Chang Sungmin
Associate Professor, Department of Korean Language Education, Inha University
Korean Language Education Research Vol. 61 No. 2 pp.225-256 (2026)
Abstract
This study examined the impact of source information indicating AI authorship on writing assessment and its effects on overall scores, text quality levels, and analytic criteria. Using a repeated-measures design, 20 Korean L1 teachers evaluated identical AI-generated texts, with and without AI authorship information. The results showed a significant decrease in scores when AI authorship was disclosed, particularly for mid-level texts and criteria involving interpretive judgment (e.g., ideas and supporting evidence; sentence and vocabulary). The qualitative findings suggest that the raters applied higher expectations and critically reinterpreted the content in terms of depth and authenticity. Thus, writing assessment is shaped by rater effects, framing effects, and algorithm aversion. This study has implications for fairness and validity in AI-era writing assessment.
Keywords
Writing assessmentartificial intelligencesource informationrater effectframing effectalgorithm aversion
