Automated Feedback vs. Human Feedback: A Study on AI-Driven Language Assessment
Keywords:
Assessment, Feedback, Arti, Human FeedbackAbstract
This study examines the comparative effectiveness of automated feedback from AI-driven language assessment tools and traditional human feedback in improving language learning outcomes. As artificial intelligence (AI) becomes increasingly integrated into educational technology, understanding its role in supporting language learners is critical. To address this, we conducted a 12-week controlled experiment with 80 EFL (English as a Foreign Language) learners, dividing participants into two groups: one receiving feedback from an AI-driven tool and the other from human instructors. Language proficiency improvements were assessed in grammar, vocabulary, and pronunciation. Findings reveal that both feedback methods significantly enhanced language skills. Human feedback proved superior in fostering gains in pronunciation and contextual understanding, while AI-driven feedback excelled in detecting and correcting grammatical errors. These results suggest that a hybrid model, blending AI and human feedback, could provide the most effective support for language learners. This research enriches the literature on AI in education and offers practical insights for incorporating AI tools into language learning curricula.
Downloads
References
Abernathy, P. (2024). Ai tools as supplementary support in language acquisition. https://doi.org/10.20944/preprints202407.2213.v1
Abuhassna, H. (2024). The information age for education via artificial intelligence and machine learning: a bibliometric and systematic literature analysis. International Journal of Information and Education Technology, 14(5), 700-711. https://doi.org/10.18178/ijiet.2024.14.5.2095
Aghaziarati, A., Nejatifar, S., & Abedi, A. (2023). Artificial Intelligence in Education: Investigating Teacher Attitudes. AI and Tech in Behavioral and Social Sciences, 1(1), 35-42. https://doi.org/10.61838/kman.aitech.1.1.6
AlAfnan, M. (2024). Artificial intelligence and language: bridging Arabic and English with technology. Journal of Ecohumanism, 3(8). https://doi.org/10.62754/joe.v3i8.4961
Alhusaiyan, E. (2024). A systematic review of current trends in artificial intelligence in foreign language learning. Saudi Journal of Language Studies, 5(1), 1-16. https://doi.org/10.1108/sjls-07-2024-0039
Alzahrani, F., & Alotaibi, H. (2024). The impact of artificial intelligence on enhancing EFL writing skills among high school students. Journal of Educational and Human Sciences, 34. https://doi.org/10.33193/jeahs.34.2024.482
Amin, M. (2023). Ai and ChatGPT in language teaching: enhancing EFL classroom support and transforming assessment techniques. International Journal of Higher Education Pedagogies, 4(4), 1-15. https://doi.org/10.33422/ijhep.v4i4.554
Belda-Medina, J., & Goddard, M. (2024). Ai-driven digital storytelling: a strategy for creating English as a foreign language (EFL) materials. International Journal of Linguistics Studies, 4(1), 40-49. https://doi.org/10.32996/ijls.2024.4.1.4
Bulut, O., & Wongvorachan, T. (2022). Feedback generation through artificial intelligence.
Byeon, H. (2023). Advances in value-based, policy-based, and deep learning-based reinforcement learning. International Journal of Advanced Computer Science and Applications, 14(8). https://doi.org/10.14569/ijacsa.2023.0140838
Chen, A., Zhang, Y., Jia, J., Liang, M., Cha, Y., & Lim, C. (2024). A systematic review and meta‐analysis of ai‐enabled assessment in language learning: design, implementation, and effectiveness. Journal of Computer Assisted Learning, 41(1). https://doi.org/10.1111/jcal.13064
Chen, Y. (2024). Enhancing language acquisition: the role of ai in facilitating effective language learning. In (pp. 593-600). https://doi.org/10.2991/978-2-38476-253-8_71
Cheung, W., & Luk, C. (2024). Implementing automated error correction and feedback loops in Kimi, a Chinese large language model.
Cohen, S., Mompelat, L., Mann, A., & Connors, L. (2024). The linguistic leap: understanding, evaluating, and integrating ai in language education. Journal of Language Teaching, 4(2), 23-31. https://doi.org/10.54475/jlt.2024.012
Davis, R. (2024). Korean in-service teachers' perceptions of implementing artificial intelligence (ai) education for teaching in schools and their ai teacher training programs. International Journal of Information and Education Technology, 14(2), 214-219. https://doi.org/10.18178/ijiet.2024.14.2.2042
Davoodifard, M., & Eskin, D. (2024). Exploring the role of generative ai in second language education: insights for instruction, learning, and assessment. Studies in Applied Linguistics and Tesol, 24(1). https://doi.org/10.52214/salt.v24i1.12863
Dewi, A., Putri, E., & Widiawati, U. (2023). Conventional written corrective feedback for EFL learners' writing skill enhancement. International Journal of Educational Best Practices, 7(2), 172. https://doi.org/10.31258/ijebp.v7n2.p172-188
Dhanapal, C., Asharudeen, N., & Alfaruque, S. (2024). Impact of artificial intelligence versus traditional instruction for language learning: a survey. World Journal of English Language, 14(2), 182. https://doi.org/10.5430/wjel.v14n2p182
Fitria, T. (2023). The use of artificial intelligence in education (AIED): can ai replace the teacher's role? Epigram, 20(2), 165-187. https://doi.org/10.32722/epi.v20i2.5711
Gupta, P., Sreelatha, C., Latha, A., Raj, S., & Singh, A. (2024). Navigating the future of education: the impact of artificial intelligence on teacher-student dynamics. EATP, 6006-6013. https://doi.org/10.53555/kuey.v30i4.2332
Hashim, H., Jamal, M., Esa, I., & Rafiq, K. (2023). Teall: technology-enhanced active language learning. Creative Education, 14(07), 1453-1462. https://doi.org/10.4236/ce.2023.147092
Huang, J. (2023). Engineering ChatGPT prompts for EFL writing classes. International Journal of Tesol Studies. https://doi.org/10.58304/ijts.20230405
Ibrahim, A. (2024). Assessing the knowledge and perception of artificial intelligence for teaching and research among lecturers in the faculties of arts in Nigeria. Journal of Global Research in Education and Social Science, 18(2), 25-33. https://doi.org/10.56557/jogress/2024/v18i28671
Ifraheem, S., Rasheed, M., & Siddiqui, A. (2024). Transforming education through artificial intelligence: personalization, engagement and predictive analytics. Journal of Asian Development Studies, 13(2), 250-266. https://doi.org/10.62345/jads.2024.13.2.22
Igbokwe, I. (2024). Artificial intelligence in educational leadership: risks and responsibilities. EJAHSS, 1(6), 3-10. https://doi.org/10.59324/ejahss.2024.1(6).01
Kim, J., Ham, Y., & Lee, S. (2024). Differences in student-ai interaction process on a drawing task: focusing on students' attitude towards ai and the level of drawing skills. Australasian Journal of Educational Technology. https://doi.org/10.14742/ajet.8859
Koka, N., Khan, A., & Jan, N. (2023). Exploring the attitudes and perceptions of foreign language lecturers on artificial intelligence-driven writing evaluation and feedback tools for improving undergraduate writing skills. Journal of Southwest Jiaotong University, 58(5). https://doi.org/10.35741/issn.0258-2724.58.5.6
Konyrova, L. (2024). The evolution of language learning: exploring ai's impact on teaching English as a second language. Eurasian Science Review, 2(2), 133-138. https://doi.org/10.63034/esr-42
Kumari, M. (2023). Perception of dental students in incorporating artificial intelligence into dental education. Journal of Advanced Sciences, 2(1), 41-45. https://doi.org/10.58935/joas.v2i1.27
Li, Y. (2024). Usability of ChatGPT in second language acquisition: capabilities, effectiveness, applications, challenges, and solutions. Studies in Applied Linguistics and Tesol, 24(1). https://doi.org/10.52214/salt.v24i1.12864
Liu, M. (2023). Exploring the application of artificial intelligence in foreign language teaching: challenges and future development. SHS Web of Conferences, 168, 03025. https://doi.org/10.1051/shsconf/202316803025
Liu, S., Wright, A., Patterson, B., Wanderer, J., Turer, R., & Nelson, S. (2023). Assessing the value of ChatGPT for clinical decision support optimization. https://doi.org/10.1101/2023.02.21.23286254
Liu, Y. (2023). A comparison of automated corrective feedback and traditional corrective feedback: a review study. The Educational Review USA, 7(9), 1365-1368. https://doi.org/10.26855/er.2023.09.024
Long, L., & Kim, M. (2024). It is like a friend to me: critical usage of automated feedback systems by self-regulating English learners in higher education. https://doi.org/10.14742/ajet.8821
Ma, W., Mat, A., Rahman, G., Hamzah, F., & Halim, R. (2024). Digital technologies in language education: a comprehensive review and analysis. Journal of Advanced Research in Applied Sciences and Engineering Technology, 41(1), 90-102. https://doi.org/10.37934/araset.41.1.90102
Mahmud, M., Mat, N., Manap, M., Zakaria, S., Tazijan, F., & Wong, S. (2023). Sustainability of educational technologies: an approach to ratio and feedback. Asian Journal of Contemporary Education, 7(2), 56-65. https://doi.org/10.55493/5052.v7i2.4895
Mohammadkarimi, E. (2024). Exploring the use of artificial intelligence in promoting English language pronunciation skills. LLT Journal: A Journal on Language and Language Teaching, 27(1), 98-115. https://doi.org/10.24071/llt.v27i1.8151
Morales, M., Amado-Salvatierra, H., Medina, J., Barchino, R., Hernández-Rizzardini, R., & Teixeira, A. (2024). Personalized feedback in massive open online courses: harnessing the power of Lang chain and OpenAI API. Electronics, 13(10), 1960. https://doi.org/10.3390/electronics13101960
Moybeka, A., Syariatin, N., Tatipang, D., Mushthoza, D., Dewi, N., & Tineh, S. (2023). Artificial intelligence and English classroom: the implications of ai toward EFL students' motivation. Edumaspul - Jurnal Pendidikan, 7(2), 2444-2454. https://doi.org/10.33487/edumaspul.v7i2.6669
Muslimin, A., Mukminatien, N., & Maria, I. (2024). Evaluating Cami AI across smart stages: students' achievement and perceptions in EFL writing instruction. Online Learning, 28(2). https://doi.org/10.24059/olj.v28i2.4246
Nurjanah, A., Salsabila, I., Azzahra, A., Rahayu, R., & Marlina, N. (2024). Artificial intelligence (AI) usage in today's teaching and learning process: a review. Syntax Idea, 6(3), 1517-1523. https://doi.org/10.46799/syntax-idea.v6i3.3126
Nurmala, I., Irianto, S., Franchisca, S., Amsa, H., & Susanti, R. (2023). Technology-enhanced language learning: a meta-analysis study on English language teaching tools. Journal on Education, 6(1), 2188-2195. https://doi.org/10.31004/joe.v6i1.3221
Ojeda-Ramirez, S., Rismanchian, S., & Doroudi, S. (2023). Learning about AI to learn about learning: artificial intelligence as a tool for metacognitive reflection.
Pack, A., & Maloney, J. (2024). Using artificial intelligence in TESOL: some ethical and pedagogical considerations. TESOL Quarterly, 58(2), 1007-1018. https://doi.org/10.1002/tesq.3320
Qin, L. (2024). Adaptive system of English-speaking learning based on artificial intelligence. Journal of Electrical Systems, 20(6s), 267-275. https://doi.org/10.52783/jes.2637
Rütti-Joy, O., Winder, G., & Biedermann, H. (2023). Building AI literacy for sustainable teacher education. Zeitschrift für Hochschulentwicklung, 18(4), 175-189. https://doi.org/10.21240/zfhe/18-04/10
Șișianu, A., & Pușcașu, A. (2024). New technologies and foreign language learning: transforming language education. Journal of Social Sciences, 7(1), 91-99. https://doi.org/10.52326/jss.utm.2024.7(1).07
Stracke, C., Chounta, I., Holmes, W., Tlili, A., & Bozkurt, A. (2023). A standardised Prisma-based protocol for systematic reviews of the scientific literature on artificial intelligence and education. Journal of Applied Learning & Teaching, 6(2). https://doi.org/10.37074/jalt.2023.6.2.38
Tiwari, H. (2024). Artificial intelligence in the classroom: revolutionizing English language teaching. Journal of English Teaching and Linguistics Studies, 6(1), 42-59. https://doi.org/10.55215/jetli.v6i1.9757
Trang, N., & Thư, P. (2024). The role of AI in improving student learning outcomes: evidence in Vietnam. International Journal of Multidisciplinary Research and Analysis, 07(06). https://doi.org/10.47191/ijmra/v7-i06-48
Vadivel, B., Shaban, A., Ahmed, Z., & Saravanan, B. (2023). Unlocking English proficiency: assessing the influence of ai-powered language learning apps on young learners' language acquisition. International Journal of English Language Education and Literature Studies (IJELLS), 2(6), 55-62. https://doi.org/10.22161/ijeel.2.6.7
Wei, L. (2023). Artificial intelligence in language instruction: impact on English learning achievement, L2 motivation, and self-regulated learning. Frontiers in Psychology, 14. https://doi.org/10.3389/fpsyg.2023.1261955
Wei, P., Wang, X., & Dong, H. (2023). The impact of automated writing evaluation on second language writing skills of Chinese EFL learners: a randomized controlled trial. Frontiers in Psychology, 14. https://doi.org/10.3389/fpsyg.2023.1249991
Wiyaka, W., Silitonga, L., Sunardi, S., & Pramudi, Y. (2024). From nervous to fluent: the impact of ai chatbot-assisted assessment on English reading anxiety and performance in Indonesia. Theory and Practice in Language Studies, 14(12), 3851-3860. https://doi.org/10.17507/tpls.1412.20
Wu, L., Li, K., Yu, M., & Lin, Y. (2024). Application of artificial intelligence in teaching English as a foreign language: progress, challenges, and trends. English Language Teaching and Linguistics Studies, 6(4), 215. https://doi.org/10.22158/eltls.v6n4p215
Xu, W., & Ouyang, F. (2022). The application of ai technologies in stem education: a systematic review from 2011 to 2021. International Journal of Stem Education, 9(1). https://doi.org/10.1186/s40594-022-00377-5
Yang, H., & Kyun, S. (2022). The current research trend of artificial intelligence in language learning: a systematic empirical literature review from an activity theory perspective. Australasian Journal of Educational Technology, 180-210. https://doi.org/10.14742/ajet.7492
Yao, Y., & Chung, K. (2024). Application of neural network and structural model in ai educational performance analysis. Sensors and Materials, 36(3), 891. https://doi.org/10.18494/sam4429
Yim, I., & Wegerif, R. (2024). Teachers' perceptions, attitudes, and acceptance of artificial intelligence (ai) educational learning tools: an exploratory study on ai literacy for young students. Future in Educational Research, 2(4), 318-345. https://doi.org/10.1002/fer3.65
Yin, S., & Goh, D. (2024). Factors influencing university students' ai use and knowledge acquisition. Proceedings of the Association for Information Science and Technology, 61(1), 1162-1164. https://doi.org/10.1002/pra2.1218
Yin, T., & Hanif, H. (2024). ELTC framework: revolutionizing MUET learning by crafting a dynamic chatbot learning experience. International Journal of Academic Research in Progressive Education and Development, 13(1). https://doi.org/10.6007/ijarped/v13-i1/20208
Yu, L., & Yu, Z. (2023). Qualitative and quantitative analyses of artificial intelligence ethics in education using VOSViewer and CitNetExplorer. Frontiers in Psychology, 14. https://doi.org/10.3389/fpsyg.2023.1061778
Yunina, O. (2023). Artificial intelligence tools in foreign language teaching in higher education institutions. The Modern Higher Education Review(8). https://doi.org/10.28925/2617-5266.2023.85
Zhou, C., & Hou, F. (2024). Can ai empower l2 education? exploring its influence on the behavioural, cognitive and emotional engagement of EFL teachers and language learners. European Journal of Education, 59(4). https://doi.org/10.1111/ejed.12750
Zou, B., Du, Y., Wang, Z., Chen, J., & Zhang, W. (2023). An investigation into artificial intelligence speech evaluation programs with automatic feedback for developing EFL learners' speaking skills. Sage Open, 13(3). https://doi.org/10.1177/21582440231193818
Zulkarnain, N., & Yunus, M. (2023). Primary teachers' perspectives on using artificial intelligence technology in English as a second language teaching and learning: a systematic review. International Journal of Academic Research in Progressive Education and Development, 12(2). https://doi.org/10.6007/ijarped/v12-i2/17119
Downloads
Additional Files
Published
Issue
Section
License
Copyright (c) 2024 Mohammad Aliakbari (Author); Pooria Barzan (Corresponding Author); Mehran Sayyadi (Author)
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
