A CORPUS-BASED ANALYSIS OF LEARNER ERRORS BEFORE AND AFTER AN AI-ASSISTED WRITING INTERVENTION: EVIDENCE FROM UNDERGRADUATE EFL STUDENTS IN HYDERABAD, SINDH, PAKISTAN

Abstract

This study is an attempt to find the effect of writing intervention with the help of Artificial Intelligence (AI) on the accuracy of undergraduate EFL writers of Hyderabad, Sindh, Pakistan for a period of eight weeks. Sixty students from four affiliated campus completed a timed pre and post argumentative writing test. Ten categories of errors were identified following the error taxonomy of the corpus and two trained raters coded the errors while 20% of the texts were double coded for reliability. The total errors per 100 words at the post-intervention sample was significantly less than the pre-intervention sample and the effect size was large. In all 10 of the categories of errors, there was a decrease. The main improvements were in word form and morphology, verb tense & aspect and subject–verb agreement. The least improvement was in punctuation and capitalisation and in the other category (residual). Article and preposition errors were both two of the most common classes of errors for both time points and there was a significant improvement on these errors. Gender, year in school and academic major were not moderators of the size of the gain, indicating that the intervention worked pretty much for everyone. The findings align with the emerging evidence that decreasing surface errors and grammatical errors in L2 writing in a single semester is meaningful. They also show that some types are not suitable for rapid remediation, particularly those to do with first language transfer. Implications for classroom practice and directions for future corpus-based intervention research in the context of the South Asian EFL context are discussed.