Email this article 
(Universitas Muhammadiyah Gresik, Indonesia)(2) Laila Rochmawati
(Politeknik Penerbangan Surabaya, Indonesia)(3) Tiara Sylvia
(Politeknik Penerbangan Medan, Indonesia)(4) Ahmad Rossydi
(Politeknik Penerbangan Makassar, Indonesia)(5) Henny Dwi Bhakti
(Universitas Muhammadiyah Gresik, Indonesia)*corresponding author
AbstractAviation English proficiency is pivotal for aviation school students to ensure secure communication in global airspace per ICAO guidelines. Conventional methods are rigid, leading to inaccurate and time-consuming evaluations that hinder training efficacy. This research develops a web-based adaptive Aviation English proficiency test integrating adaptive algorithms like Item Response Theory and dynamic assessment to enhance aviation education outcomes. Using a mixed-methods framework with the ADDIE model and quantitative experimental approach, an explanatory sequential design with non-equivalent control group was employed, involving needs assessment, prototype development, validation, and implementation. The sample included 141 aviation school students. Data from pre/post-tests were analyzed via SPSS. The findings showed that i) the web-based test is valid and feasible as an assessment tool with a validation score of 89.5%; ii) student proficiency levels are significantly improved before and after using the adaptive system (paired t-test: mean rise from 72.6 to 91.4, t=-14.28, p=0.000 <0.05); iii) dynamic assessment positively impacts learning outcomes following implementation (32% uplift, ?=0.61, p<0.01); and iv) there is a significant difference between experimental and control groups in evaluation efficiency (independent t-test: 25% higher for experimental, t=10.52, p=0.000 <0.05). These affirm the test's efficacy, recommending broader adoption for refined aviation training. KeywordsAviation English Proficiency Test Adaptive Algorithms Dynamic Assessment Item Response Theory
|
DOIhttps://doi.org/10.29099/ijair.v9i1.1.1620 |
Article metrics10.29099/ijair.v9i1.1.1620 Abstract views : 100 | PDF views : 26 |
Cite |
Full Text Download
|
References
References
A. Fatima and S. Attila, “Enhancing Aviation Safety through Effective English Language Communication under the ICAO Requirements: Regulatory Challenges,” J. East Asia Int. Law, vol. 18, no. 1, pp. 97–121, May 2025, doi: 10.14330/jeail.2025.18.1.05.
L. Clark and G. J. Williams, “English Language Proficiency in Radiotelephony: A survey about its effect on the safety and efficiency of aviation,” Espec., vol. 41, no. 4, Oct. 2020, doi: 10.23925/2318-7115.2020v41i4a9.
L. Dai and J. Zhang, “A Study of Pilots’ English Proficiency Development Based on ICAO LPRs,” 2024, pp. 563–569. doi: 10.2991/978-2-38476-259-0_60.
H. Hamzah, “Clarity and pronunciation of ab-initio air traffic controller,” Aviation, vol. 25, no. 4, pp. 252–261, Dec. 2021, doi: 10.3846/aviation.2021.15906.
Y. Kim, “The challenges of radiotelephony communication and effective training approaches: A study of Korean pilots and air traffic controllers,” English Specif. Purp., vol. 72, pp. 26–39, Oct. 2023, doi: 10.1016/j.esp.2023.07.001.
“The vocabulary of aviation radiotelephony communication in simulator emergencies and the contradictions in air traffic controller beliefs about language use,” Open Access Te Herenga Waka-Victoria University of Wellington, 2021. doi: 10.26686/wgtn.17152022.
H. Hamzah, P. Krish, and A. Hamat, “Communication Competence in Pilot-Controller Interactions: Repair and Accommodation Strategies,” GEMA Online® J. Lang. Stud., vol. 22, no. 3, pp. 176–195, Aug. 2022, doi: 10.17576/gema-2022-2203-10.
A. Van Moere, M. Suzuki, R. Downey, and J. Cheng, “Implementing ICAO language proficiency requirements in the Versant Aviation English Test,” Aust. Rev. Appl. Linguist., vol. 32, no. 3, pp. 27.1-27.17, Jan. 2009, doi: 10.2104/aral0927.
A. Huhta, “An analysis of the quality of English testing for aviation purposes in Finland,” Aust. Rev. Appl. Linguist., vol. 32, no. 3, pp. 26.1-26.14, Jan. 2009, doi: 10.2104/aral0926.
M. Treadaway and J. Read, “Setting standards for a diagnostic test of aviation English for student pilots,” Lang. Test., vol. 41, no. 3, pp. 557–583, Jul. 2024, doi: 10.1177/02655322231224051.
K. J. Ruskin, C. Corvin, S. Rice, G. Richards, S. R. Winter, and A. Clebone Ruskin, “Alarms, alerts, and warnings in air traffic control: An analysis of reports from the Aviation Safety Reporting System,” Transp. Res. Interdiscip. Perspect., vol. 12, p. 100502, Dec. 2021, doi: 10.1016/j.trip.2021.100502.
V. De Luca, C. Pascarelli, M. Colucci, P. Afrune, A. Corallo, and G. Avanzini, “A Platform for Safe Operations of Unmanned Aircraft Systems in Critical Areas,” Engineering, vol. 49, pp. 314–331, Jun. 2025, doi: 10.1016/j.eng.2025.02.004.
M. Bieswanger, M. C. de A. Prado, and J. Roberts, “Pilot training and English as a lingua franca: some implications for the design of Aviation English for ab initio flight training courses,” Espec., vol. 41, no. 4, Oct. 2020, doi: 10.23925/2318-7115.2020v41i4a7.
M. C. de A. Prado, “Pragmática do inglês aeronáutico,” Rev. Estud. da Ling., vol. 29, no. 2, pp. 1381–1414, Oct. 2024, doi: 10.17851/2237-2083.29.2.1381-1414.
P. Tosqui-Lucks and A. L. B. de C. e Silva, “Aeronautical English: Investigating the nature of this specific language in search of new heights,” Espec., vol. 41, no. 3, Oct. 2020, doi: 10.23925/2318-7115.2020v41i3a2.
A. Pacheco and A. C. De Moraes Garcia, “Language issues of Brazilian pilots regarding structure: A comparative study between student pilots and SDEA test takers,” Espec., vol. 41, no. 3, Oct. 2020, doi: 10.23925/2318-7115.2020v41i3a6.
W. T.-K. Chan, W.-C. Li, and G. Braithwaite, “Pilots’ training backgrounds affecting the attribution of event causal factors and airline safety management,” J. Air Transp. Manag., vol. 125, p. 102786, May 2025, doi: 10.1016/j.jairtraman.2025.102786.
E. Gkintoni, H. Antonopoulou, A. Sortwell, and C. Halkiopoulos, “Challenging Cognitive Load Theory: The Role of Educational Neuroscience and Artificial Intelligence in Redefining Learning Efficacy,” Brain Sci., vol. 15, no. 2, p. 203, Feb. 2025, doi: 10.3390/brainsci15020203.
C. Halkiopoulos and E. Gkintoni, “Leveraging AI in E-Learning: Personalized Learning and Adaptive Assessment through Cognitive Neuropsychology—A Systematic Analysis,” Electronics, vol. 13, no. 18, p. 3762, Sep. 2024, doi: 10.3390/electronics13183762.
S. Khamkaew and A. Trussat, “The Effects of Using the Zone of Proximal Development (ZPD) on the Fundamental English Classes of Thai EFL Learners,” Int. J. English Lang. Teach., vol. 12, no. 1, pp. 46–64, Jan. 2024, doi: 10.37745/ijelt.13/vol12n14664.
G. Akduman and M. Öksüz, “Evaluation of the need for sustainability education in aviation management undergraduate training: a case study of türkiye,” Sos. Ara?t?rmalar ve Yönetim Derg., no. 1, pp. 66–77, Sep. 2025, doi: 10.35375/sayod.1780750.
C. St?Onge, K. Ouellet, S. Lakhal, T. Dubé, and M. Marceau, “COVID?19 as the tipping point for integrating e?assessment in higher education practices,” Br. J. Educ. Technol., vol. 53, no. 2, pp. 349–366, Mar. 2022, doi: 10.1111/bjet.13169.
S. Sahni, S. Verma, and R. P. S. Kaurav, “Understanding digital transformation challenges for online learning and teaching in higher education institutions: a review and research framework,” Benchmarking An Int. J., vol. 32, no. 5, pp. 1487–1521, May 2025, doi: 10.1108/BIJ-04-2022-0245.
A. Kushki and H. Nassaji, “L2 Reading Assessment from a Sociocultural Theory Perspective: The Contributions of Dynamic Assessment,” Educ. Sci., vol. 14, no. 4, p. 342, Mar. 2024, doi: 10.3390/educsci14040342.
Intikhab Ahmed Panhwar, Muhammad Usman, Faiza Panhwar, and Tahreem Akhter Surahio*, “Exploring the Impact of Zone of Proximal Development and Scaffolding in Second Language Acquisition: A Comparative Study of Vygotskian and Freirean Approaches,” Soc. Sci. Rev. Arch., vol. 3, no. 1, pp. 1166–1176, Feb. 2025, doi: 10.70670/sra.v3i1.416.
N.-V. Sarmiento-Campos et al., “A Look at Vygotsky’s Sociocultural Theory (SCT): The Effectiveness of Scaffolding Method on EFL Learners’ Speaking Achievement,” Educ. Res. Int., vol. 2022, pp. 1–12, Dec. 2022, doi: 10.1155/2022/3514892.
D. Tzuriel, “Tzuriel’s Dynamic Assessment Instruments for Young Children,” 2021, pp. 99–166. doi: 10.1007/978-3-030-75692-5_6.
W. Ren, S. Li, and X. Lü, “A Meta-analysis of the Effectiveness of Second Language Pragmatics Instruction,” Appl. Linguist., vol. 44, no. 6, pp. 1010–1029, Jan. 2023, doi: 10.1093/applin/amac055.
Y. Shang, V. Aryadoust, and Z. Hou, “A Meta-Analysis of the Reliability of Second Language Listening Tests (1991–2022),” Brain Sci., vol. 14, no. 8, p. 746, Jul. 2024, doi: 10.3390/brainsci14080746.
F. Shakki, “Meta-analysis as an emerging trend to scrutinize the effectiveness of L2 pragmatics instruction,” Front. Psychol., vol. 13, Nov. 2022, doi: 10.3389/fpsyg.2022.1016661.
P. T. Thu Nguyet and T. P. Cam Tu, “Overview of Computerized Adaptive Testing,” in 2023 IEEE/ACIS 8th International Conference on Big Data, Cloud Computing, and Data Science (BCD), IEEE, Dec. 2023, pp. 181–185. doi: 10.1109/BCD57833.2023.10466281.
A. Huda, F. Firdaus, D. Irfan, Y. Hendriyani, A. Almasri, and M. Sukmawati, “Optimizing Educational Assessment: The Practicality of Computer Adaptive Testing (CAT) with an Item Response Theory (IRT) Approach,” JOIV Int. J. Informatics Vis., vol. 8, no. 1, p. 473, Mar. 2024, doi: 10.62527/joiv.8.1.2217.
L. Xu, Z. Jiang, Y. Han, H. Liang, and J. Ouyang, “Developing Computerized Adaptive Testing for a National Health Professionals Exam: An Attempt from Psychometric Simulations,” Perspect. Med. Educ., vol. 12, no. 1, Oct. 2023, doi: 10.5334/pme.855.
A. Cobbinah And S. Ntumi, “Difficulty, Discrimination and Pseudo - Guessing Indices of West African Examinations Council Core Mathematics Multiple Choice Items: Theoretical and Practical Implications of Using Item Response Theory,” J. Res. Educ. Sci., vol. 13, no. 15, p. 51, Dec. 2022, doi: 10.14505/jres.v13.15.06.
Á. I. Berrío, J. Gómez-Benito, and E. M. Arias-Patiño, “Developments and trends in research on methods of detecting differential item functioning,” Educ. Res. Rev., vol. 31, p. 100340, Nov. 2020, doi: 10.1016/j.edurev.2020.100340.
A. Beatty, E. Berkhout, L. Bima, M. Pradhan, and D. Suryadarma, “Schooling progress, learning reversal: Indonesia’s learning profiles between 2000 and 2014,” Int. J. Educ. Dev., vol. 85, p. 102436, Sep. 2021, doi: 10.1016/j.ijedudev.2021.102436.
M. A. Ayanwale, “Harnessing IRT and CAT for Next-Generation Educational Assessment in the Fifth Industrial Revolution,” J. Comput. Adapt. Test. Africa, vol. 2, pp. 49–68, Dec. 2023, doi: 10.71291/jocatia.v2i.27.
L. Dwahdh and N. Alshraifin, “The impact of computerized adaptive test termination rules on accuracy across different ability estimation methods,” Eurasia J. Math. Sci. Technol. Educ., vol. 21, no. 1, p. em2571, Jan. 2025, doi: 10.29333/ejmste/15897.
N. Ebenbeck and M. Gebhardt, “Differential Performance of Computerized Adaptive Testing in Students With and Without Disabilities – A Simulation Study,” J. Spec. Educ. Technol., vol. 39, no. 4, pp. 481–490, Dec. 2024, doi: 10.1177/01626434241232117.
M. A. Petersen, H. Vachon, J. M. Giesinger, and M. Groenvold, “Development of standard computerised adaptive test (CAT) settings for the EORTC CAT Core,” Qual. Life Res., vol. 33, no. 4, pp. 951–961, Apr. 2024, doi: 10.1007/s11136-023-03576-x.
“IRT-Based Methods,” in Differential Item Functioning, 2455 Teller Road, Thousand Oaks California 91320 United States of America: SAGE Publications, Inc., 2009, pp. 39–59. doi: 10.4135/9781412993913.d40.
M. C.A. Prado and P. Tosquil Lucks, “Designing the Radiotelephony Plain English Corpus (RTPEC): A specialized spoken English language corpus towards a description of aeronautical communications in non-routine situations,” Res. Corpus Linguist., vol. 7, pp. 113–128, 2019, doi: 10.32714/ricl.07.06.
R. Dinçer, N. Dinçer, And O. Guksu, “An interactive conversation with a chatbot: Does ChatGPT know standard phraseology in aviation English?,” Lit. Trek, vol. 9, no. 2, pp. 24–41, Dec. 2023, doi: 10.47216/literacytrek.1382543.
S. Chen, W. Pan, Y. Wang, S. Chen, and X. Wang, “Research on the Method of Air Traffic Control Instruction Keyword Extraction Based on the Roberta-Attention-BiLSTM-CRF Model,” Aerospace, vol. 12, no. 5, p. 376, Apr. 2025, doi: 10.3390/aerospace12050376.
W. Zhan, Y. Shi, and C. Lin, “The impact of IRT and IDT on metaverse adoption in tourism: The moderating role of personal innovativeness among China’s generation Z,” PLoS One, vol. 20, no. 6, p. e0327239, Jun. 2025, doi: 10.1371/journal.pone.0327239.
E.-Y. Mun, Y. Huo, H. R. White, S. Suzuki, and J. de la Torre, “Multivariate Higher-Order IRT Model and MCMC Algorithm for Linking Individual Participant Data From Multiple Studies,” Front. Psychol., vol. 10, Jun. 2019, doi: 10.3389/fpsyg.2019.01328.
S. I. Shakeel, M. A. Al Mamun, and M. F. A. Haolader, “Instructional design with ADDIE and rapid prototyping for blended learning: validation and its acceptance in the context of TVET Bangladesh,” Educ. Inf. Technol., vol. 28, no. 6, pp. 7601–7630, Jun. 2023, doi: 10.1007/s10639-022-11471-0.
L. Wambugu and N. Njoroge, “The search for understanding of mixed method research among graduate students: a case of learners in the school of continuing and distance education, university of Nairobi, Kenya,” Qual. Quant., vol. 56, no. 2, pp. 843–855, Apr. 2022, doi: 10.1007/s11135-021-01150-6.
S. Al-Amri, S. Hamid, N. F. M. Noor, and A. Gani, “A framework for designing interactive mobile training course content using augmented reality,” Multimed. Tools Appl., vol. 82, no. 20, pp. 30491–30541, Aug. 2023, doi: 10.1007/s11042-023-14561-4.
M. Asoodar, F. Janesarvatan, H. Yu, and N. de Jong, “Theoretical foundations and implications of augmented reality, virtual reality, and mixed reality for immersive learning in health professions education,” Adv. Simul., vol. 9, no. 1, p. 36, Sep. 2024, doi: 10.1186/s41077-024-00311-5.
A. G. Spatioti, I. Kazanidis, and J. Pange, “A Comparative Study of the ADDIE Instructional Design Model in Distance Education,” Information, vol. 13, no. 9, p. 402, Aug. 2022, doi: 10.3390/info13090402.
ICAO, Manual on the Implementation of ICAO Language Proficiency Requirements. 2010.
P. SHAWCROSS, New guidelines for aviation English training programmes. 2009.
J. Baumgartner, N. Ruettgers, A. Hasler, A. Sonderegger, and J. Sauer, “Questionnaire experience and the hybrid System Usability Scale: Using a novel concept to evaluate a new instrument,” Int. J. Hum. Comput. Stud., vol. 147, p. 102575, Mar. 2021, doi: 10.1016/j.ijhcs.2020.102575.
S. Wei and Y. Shin, “Analyzing the effects of computer-assisted dynamic assessment on L2 writing tasks for Chinese learners,” Cogent Educ., vol. 12, no. 1, Dec. 2025, doi: 10.1080/2331186X.2025.2510082.
C. C. M. Goh and V. Aryadoust, “Developing and assessing second language listening and speaking: Does AI make it better?,” Annu. Rev. Appl. Linguist., vol. 45, pp. 179–199, Mar. 2025, doi: 10.1017/S0267190525100111.
E.-R. Ro, K.-O. An, M.-J. Lim, S.-Y. Lee, D.-A. Kim, and S.-D. Eun, “The Development of an Interview Questionnaire and Guide for the Sustainable Use of Assistive Devices among the Disabled in Korea,” Sustainability, vol. 15, no. 17, p. 12812, Aug. 2023, doi: 10.3390/su151712812.
Z. Xu, Y. Wang, and Y. Qian, “The design and application of in-depth interviews in primary care research,” Chinese Gen. Pract. J., vol. 2, no. 2, p. 100062, Jun. 2025, doi: 10.1016/j.cgpj.2025.100062.
B. C. Lee, A. Pandian, S. Rethinasamy, and D. A. L. Tan, “Effects of PWIM in the ESL Classroom: Vocabulary Knowledge Development Among Primary Malaysian Learners,” 3L Southeast Asian J. English Lang. Stud., vol. 25, no. 4, pp. 179–197, Dec. 2019, doi: 10.17576/3L-2019-2504-11.
S. Thukral, S. Kovac, and M. Paturu, “t-test,” in Translational Interventional Radiology, Elsevier, 2023, pp. 139–143. doi: 10.1016/B978-0-12-823026-8.00104-8.
C. M. Hazell, L. Chapman, S. F. Valeix, P. Roberts, J. E. Niven, and C. Berry, “Understanding the mental health of doctoral researchers: a mixed methods systematic review with meta-analysis and meta-synthesis,” Syst. Rev., vol. 9, no. 1, p. 197, Dec. 2020, doi: 10.1186/s13643-020-01443-1.
L. Eyal and R. Jacobson, “Enhancing language learning: Design principles for building effective AI platforms to boost spoken english for matriculation,” Eur. J. Open, Distance E-Learning, vol. 27, no. 2, p. 1, Jun. 2025, doi: 10.65043/eurodl.148.
X. Sun, “Ten years later: Reexamining the TESOL Technology Standards for Language Teachers,” TESOL J., vol. 13, no. 4, Dec. 2022, doi: 10.1002/tesj.684.
M. van Haastrecht, M. Haas, M. Brinkhuis, and M. Spruit, “Understanding validity criteria in technology-enhanced learning: A systematic literature review,” Comput. Educ., vol. 220, p. 105128, Oct. 2024, doi: 10.1016/j.compedu.2024.105128.
I. Backfisch, R. Scherer, J. Schneider, and A. Lachner, “How valid, really? A meta-analysis of the validity evidence of Technological Pedagogical and Content Knowledge (TPACK) self-report assessments,” Educ. Res. Rev., vol. 49, p. 100718, Nov. 2025, doi: 10.1016/j.edurev.2025.100718.
A. Bossman and S. K. Agyei, “Technology and instructor dimensions, e-learning satisfaction, and academic performance of distance students in Ghana,” Heliyon, vol. 8, no. 4, p. e09200, Apr. 2022, doi: 10.1016/j.heliyon.2022.e09200.
J. Braman, A. Brown, and M. J. Richards, Eds., Reshaping Learning with Next Generation Educational Technologies. in Advances in Educational Technologies and Instructional Design. IGI Global, 2024. doi: 10.4018/979-8-3693-1310-7.
M. S. Alharbi, “Digital transformation and EFL anxiety: A mixed-methods study on the impact of online learning on the language acquisition psychology of Saudi Arabian students,” Soc. Sci. Humanit. Open, vol. 12, p. 101721, 2025, doi: 10.1016/j.ssaho.2025.101721.
J. Abdulbaqi and G. O. Uzun, “Effect of the Students’ Test Anxiety Prevention Program on Students’ Experiences of Test Anxiety: A Mixed-Methods Experiment,” Sage Open, vol. 14, no. 4, Oct. 2024, doi: 10.1177/21582440241296609.
W. Chaiyasit, K. Chomsuwan, and S. Chanchalor, “Hybrid Teaching Using Problem-Based Learning to Promote Self-Directed Learning Abilities of Students during the COVID-19 Pandemic,” Int. J. Learn. Teach. Educ. Res., vol. 22, no. 8, pp. 1–22, Aug. 2023, doi: 10.26803/ijlter.22.8.1.
Y. Zhao, S. Mohd Yusof, and M. Hou, “Developing through Merits and Demerits: A Literature Review of Online Classroom, Teaching Motivation and Teaching Methods in China Primary School,” Int. J. Learn. Teach. Educ. Res., vol. 22, no. 8, pp. 183–202, Aug. 2023, doi: 10.26803/ijlter.22.8.11.
L. Darling-Hammond, L. Flook, C. Cook-Harvey, B. Barron, and D. Osher, “Implications for educational practice of the science of learning and development,” Appl. Dev. Sci., vol. 24, no. 2, pp. 97–140, Apr. 2020, doi: 10.1080/10888691.2018.1537791.
G. K. Tsismalidou and B. K. Kondilis, “The Effects of Stress and Fatigue on Levels of Anxiety in Pilots: An Aviation Industry Sample,” J. Organ. Psychol., vol. 24, no. 2, Jul. 2024, doi: 10.33423/jop.v24i2.7084.
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
________________________________________________________
The International Journal of Artificial Intelligence Research
Organized by: Prodi Teknik Informatika Fakultas Teknologi Bisnis dan Sains
Published by: Universitas Dharma Wacana
Jl. Kenanga No. 03 Mulyojati 16C Metro Barat Kota Metro Lampung
Email: jurnal.ijair@gmail.com
This work is licensed under Creative Commons Attribution-ShareAlike 4.0 International License.