Email this article 
(Sistem Informasi, Fakultas Teknik, Universitas Muria Kudus, Indonesia)(2) Soni Adiyono
(Sistem Informasi, Fakultas Teknik, Universitas Muria Kudus, Indonesia)*corresponding author
AbstractPrediction of student academic achievement is a very important research area; this can be seen from the many researchers who conduct research in this area. To make predictions, a machine learning model is needed. Along with their parameters, the majority of machine learning models have associated hyperparameters. However, knowing the right mix of hyperparameters is essential for robust model performance. A methodical procedure called hyperparameter optimization (HPO) aids in determining the appropriate values for them. In this study we compared four hyperparameters tuning techniques, namely HyperOpt, Random Search, Optuna and Grid Search. The results of the hyperparameters from each of these techniques are then used in machine learning algorithms to predict student academic achievement. Validation uses the 5-fold cross validation method while performance testing uses Mean absolute error. From the experimental results it was found that the hyperparameter technique The best method for predicting student academic achievement in machine learning models is gridsearchcv.
KeywordsHyperparameters; Gradient Boosting Tree; Grid Search; Random Search; Optuna
|
DOIhttps://doi.org/10.29099/ijair.v8i1.1.1214 |
Article metrics10.29099/ijair.v8i1.1.1214 Abstract views : 1128 | PDF views : 445 |
Cite |
Full Text Download
|
References
C. Romero, M. Ventura, Sebastian Pechenizkiy, and R. S. J. . Baker, Handbook of Educational Data Mining, 1st ed. United States of America: Springer US, 2010.
R. O. Aluko, E. I. Daniel, O. Shamsideen Oshodi, C. O. Aigbavboa, and A. O. Abisuga, “Towards reliable prediction of academic performance of architecture students using data mining techniques,” J. Eng. Des. Technol., vol. 16, no. 3, pp. 385–397, 2018, doi: 10.1108/JEDT-08-2017-0081.
H. Karalar, C. Kapucu, and H. Gürüler, “Predicting students at risk of academic failure using ensemble model during pandemic in a distance learning system,” Int. J. Educ. Technol. High. Educ., vol. 18, no. 1, 2021, doi: 10.1186/s41239-021-00300-y.
R. Conijn, C. Snijders, A. Kleingeld, and U. Matzat, “Predicting student performance from LMS data: A comparison of 17 blended courses using moodle LMS,” IEEE Trans. Learn. Technol., vol. 10, no. 1, pp. 17–29, 2017, doi: 10.1109/TLT.2016.2616312.
S. Helal et al., “Predicting academic performance by considering student heterogeneity,” Knowledge-Based Syst., vol. 161, no. December 2017, pp. 134–146, 2018, doi: 10.1016/j.knosys.2018.07.042.
M. Arifin, Widowati, Farikhin, A. Wibowo, and B. Warsito, “Comparative Analysis on Educational Data Mining Algorithm to Predict Academic Performance,” Proc. - 2021 Int. Semin. Appl. Technol. Inf. Commun. IT Oppor. Creat. Digit. Innov. Commun. within Glob. Pandemic, iSemantic 2021, pp. 173–178, 2021, doi: 10.1109/iSemantic52711.2021.9573185.
L. W. Santoso and Yulia, “Predicting student performance in higher education using multi-regression models,” Telkomnika (Telecommunication Comput. Electron. Control., vol. 18, no. 3, pp. 1354–1360, 2020, doi: 10.12928/TELKOMNIKA.v18i3.14802.
P. Probst, A. L. Boulesteix, and B. Bischl, “Tunability: Importance of hyperparameters of machine learning algorithms,” J. Mach. Learn. Res., vol. 20, pp. 1–32, 2019.
C. Qi, A. Fourie, and X. Zhao, “Back-Analysis Method for Stope Displacements Using Gradient-Boosted Regression Tree and Firefly Algorithm,” J. Comput. Civ. Eng., vol. 32, no. 5, Sep. 2018, doi: 10.1061/(ASCE)CP.1943-5487.0000779.
M. T. Young, J. Hinkle, A. Ramanathan, and R. Kannan, “HyperSpace: Distributed Bayesian Hyperparameter Optimization,” Proc. - 2018 30th Int. Symp. Comput. Archit. High Perform. Comput. SBAC-PAD 2018, no. 1, pp. 339–347, 2019, doi: 10.1109/CAHPC.2018.8645954.
T. Chen and C. Guestrin, “XGBoost: A scalable tree boosting system,” Proc. ACM SIGKDD Int. Conf. Knowl. Discov. Data Min., vol. 13-17-Augu, pp. 785–794, 2016, doi: 10.1145/2939672.2939785.
E. Elgeldawi, A. Sayed, A. R. Galal, and A. M. Zaki, “Hyperparameter tuning for machine learning algorithms used for arabic sentiment analysis,” Informatics, vol. 8, no. 4, pp. 1–21, 2021, doi: 10.3390/informatics8040079.
R. G. Mantovani, A. L. D. Rossi, E. Alcobaça, J. Vanschoren, and A. C. P. L. F. de Carvalho, “A meta-learning recommender system for hyperparameter tuning: Predicting when tuning improves SVM classifiers,” Inf. Sci. (Ny)., vol. 501, pp. 193–221, Oct. 2019, doi: 10.1016/j.ins.2019.06.005.
H. J. P. Weerts, A. C. Mueller, and J. Vanschoren, “Importance of Tuning Hyperparameters of Machine Learning Algorithms,” 2020.
A. H. Victoria and G. Maragatham, “Automatic tuning of hyperparameters using Bayesian optimization,” Evol. Syst., vol. 12, no. 1, pp. 217–223, 2021, doi: 10.1007/s12530-020-09345-2.
J. Zhang, Q. Wang, and W. Shen, “Hyper-parameter optimization of multiple machine learning algorithms for molecular property prediction using hyperopt library,” Chinese J. Chem. Eng., vol. 52, pp. 115–125, 2022, doi: 10.1016/j.cjche.2022.04.004.
S. Shekhar, A. Bansode, and A. Salim, “A Comparative study of Hyper-Parameter Optimization Tools,” 2021 IEEE Asia-Pacific Conf. Comput. Sci. Data Eng. CSDE 2021, 2021, doi: 10.1109/CSDE53843.2021.9718485.
J. Rijsdijk, L. Wu, G. Perin, and S. Picek, “Reinforcement learning for hyperparameter tuning in deep learning-based side-channel analysis,” IACR Trans. Cryptogr. Hardw. Embed. Syst., vol. 2021, no. 3, pp. 677–707, 2021, doi: 10.46586/tches.v2021.i3.677-707.
P. Schratz, J. Muenchow, E. Iturritxa, J. Richter, and A. Brenning, “Hyperparameter tuning and performance assessment of statistical and machine-learning algorithms using spatial data,” Ecol. Modell., vol. 406, no. April 2018, pp. 109–120, 2019, doi: 10.1016/j.ecolmodel.2019.06.002.
J. Wong, T. Manderson, M. Abrahamowicz, D. L. Buckeridge, and R. Tamblyn, “Can Hyperparameter Tuning Improve the Performance of a Super Learner?: A Case Study,” Epidemiology, vol. 30, no. 4, pp. 521–531, 2019, doi: 10.1097/EDE.0000000000001027.
P. Probst, M. N. Wright, and A. Boulesteix, “Hyperparameters and tuning strategies for random forest,” WIREs Data Min. Knowl. Discov., vol. 9, no. 3, May 2019, doi: 10.1002/widm.1301.
M. MacKay, P. Vicol, J. Lorraine, D. Duvenaud, and R. Grosse, “Self-tuning networks: Bilevel optimization of hyperparameters using structured best-response functions,” 7th Int. Conf. Learn. Represent. ICLR 2019, vol. 1, no. 1, pp. 1–25, 2019.
J. Lorraine, P. Vicol, and D. Duvenaud, “Optimizing Millions of Hyperparameters by Implicit Differentiation,” vol. 108, 2019.
E. Duarte and J. Wainer, “Empirical comparison of cross-validation and internal metrics for tuning SVM hyperparameters,” Pattern Recognit. Lett., vol. 88, pp. 6–11, 2017, doi: 10.1016/j.patrec.2017.01.007.
S. Putatunda and K. Rama, “A Modified Bayesian Optimization based Hyper-Parameter Tuning Approach for Extreme Gradient Boosting,” 2019 15th Int. Conf. Inf. Process. Internet Things, ICINPRO 2019 - Proc., 2019, doi: 10.1109/ICInPro47689.2019.9092025.
H. Ma, X. Yang, J. Mao, and H. Zheng, “The Energy Efficiency Prediction Method Based on Gradient Boosting Regression Tree,” 2nd IEEE Conf. Energy Internet Energy Syst. Integr. EI2 2018 - Proc., vol. 1, no. 4, 2018, doi: 10.1109/EI2.2018.8581904.
P. Datta, P. Das, and A. Kumar, “Hyper parameter tuning based gradient boosting algorithm for detection of diabetic retinopathy: an analytical review,” Bull. Electr. Eng. Informatics, vol. 11, no. 2, pp. 814–824, 2022, doi: 10.11591/eei.v11i2.3559.
Z. M. Alhakeem, Y. M. Jebur, S. N. Henedy, H. Imran, L. F. A. Bernardo, and H. M. Hussein, “Prediction of Ecofriendly Concrete Compressive Strength Using Gradient Boosting Regression Tree Combined with GridSearchCV Hyperparameter-Optimization Techniques,” Materials (Basel)., vol. 15, no. 21, p. 7432, 2022, doi: 10.3390/ma15217432.
T. Akiba, S. Sano, T. Yanase, T. Ohta, and M. Koyama, “Optuna : A Next-generation Hyperparameter Optimization Framework,” in Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Jul. 2019, pp. 2623–2631. doi: 10.1145/3292500.3330701.
J. Bergstra, B. Komer, C. Eliasmith, D. Yamins, and D. D. Cox, “Hyperopt: A Python library for model selection and hyperparameter optimization,” Comput. Sci. Discov., vol. 8, no. 1, 2015, doi: 10.1088/1749-4699/8/1/014008.
P. Liashchynskyi and P. Liashchynskyi, “Grid Search, Random Search, Genetic Algorithm: A Big Comparison for NAS,” no. 2017, pp. 1–11, 2019.
L. Villalobos-Arias, C. Quesada-López, J. Guevara-Coto, A. Mart’inez, and M. Jenkins, “Evaluating Hyper-Parameter Tuning Using Random Search in Support Vector Machines for Software Effort Estimation,” in Proceedings of the 16th ACM International Conference on Predictive Models and Data Analytics in Software Engineering, 2020, pp. 31–40. doi: 10.1145/3416508.3417121.
L. Villalobos-Arias and C. Quesada-López, “Comparative study of random search hyper-parameter tuning for software effort estimation,” in Proceedings of the 17th International Conference on Predictive Models and Data Analytics in Software Engineering, Aug. 2021, pp. 21–29. doi: 10.1145/3475960.3475986.
S. Putatunda and K. Rama, “A Comparative Analysis of Hyperopt as Against Other Approaches for Hyper-Parameter Optimization of XGBoost,” in Proceedings of the 2018 International Conference on Signal Processing and Machine Learning - SPML ’18, 2018, pp. 6–10. doi: 10.1145/3297067.3297080.
J. Joy and M. P. Selvan, “A comprehensive study on the performance of different Multi-class Classification Algorithms and Hyperparameter Tuning Techniques using Optuna,” Proc. Int. Conf. Comput. Commun. Secur. Intell. Syst. IC3SIS 2022, 2022, doi: 10.1109/IC3SIS54991.2022.9885695.
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
________________________________________________________
The International Journal of Artificial Intelligence Research
Organized by: Prodi Teknik Informatika Fakultas Teknologi Bisnis dan Sains
Published by: Universitas Dharma Wacana
Jl. Kenanga No. 03 Mulyojati 16C Metro Barat Kota Metro Lampung
Email: jurnal.ijair@gmail.com
This work is licensed under Creative Commons Attribution-ShareAlike 4.0 International License.