Leakage-Aware Machine Learning for Imbalanced Network Intrusion Detection: A Comparative Study of Classical and Ensemble Classifiers
Abstract
Network intrusion detection requires reliable machine learning models capable of identifying malicious traffic while maintaining valid generalization performance. However, class imbalance and cross-split feature overlap may affect the reliability of model evaluation. This study proposes a leakage-aware evaluation framework for binary network intrusion detection using the UNSW-NB15 dataset. Logistic Regression, Linear Support Vector Machine, Random Forest, and class-weighted variants of the two linear classifiers were evaluated using official and leakage-controlled test sets. A cross-split audit identified 1,302 exact overlapping feature patterns affecting 8,541 test records (10.37%), which were excluded to construct a leakage-controlled test set of 73,791 records. All classifiers exhibited performance reductions after overlap control, indicating differences between conventional and stricter generalization estimates. Random Forest achieved the best leakage-controlled performance, with 86.23% accuracy, 88.11% F1-score, 85.88% Macro F1-score, 97.68% ROC-AUC, and 97.87% average precision. It also achieved a 98.37% attack detection rate with a 1.63% false negative rate. Class weighting improved the balanced performance of both linear classifiers. These findings demonstrate the importance of jointly considering class imbalance and train-test independence when evaluating machine-learning-based intrusion detection systems.
Article Discovery & Scholarly Lookup
DOIAuthor Biographies
Department of Informatics Engineering, Faculty of Engineering and Computer Science, Universitas Indraprasta PGRI, South Jakarta City, Special Capital Region of Jakarta, Indonesia.
Department of Informatics Engineering, Faculty of Engineering and Computer Science, Universitas Indraprasta PGRI, South Jakarta City, Special Capital Region of Jakarta, Indonesia.
Department of Informatics Engineering, Faculty of Engineering and Computer Science, Universitas Indraprasta PGRI, South Jakarta City, Special Capital Region of Jakarta, Indonesia.
How to Cite
License & Copyright
This work is licensed under a Creative Commons Attribution 4.0 International License.
This is an open-access journal. All articles are distributed under the terms of the Creative Commons Attribution 4.0 International License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Creative Commons Attribution 4.0 International (CC BY 4.0)Under this license, you are free to:
- Share: copy, distribute, and transmit the work in any medium or format.
- Adapt: remix, transform, and build upon the work for any purpose, even commercially.
You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
Copyright and Retention Copyright & Retention Authors retain copyright without restrictions and grant this journal the right of first publication under an open-access model. The journal retains non-exclusive publishing rights for archiving, indexing, and scholarly dissemination.References
Total: 16 References- Ahmad, R., Alsmadi, I., Al-Hamdani, W. A., & Tawalbeh, L. (2022). A deep learning ensemble approach to detecting unknown network attacks. Journal of Information Security and Applications, 67, 103196. https://doi.org/10.1016/j.jisa.2022.103196
- Ahmed, A., Asim, M., Ullah, I., Zainulabidin, & Ateya, A. A. (2024). An optimized ensemble model with advanced feature selection for network intrusion detection. PeerJ Computer Science, 10, e2472. https://doi.org/10.7717/peerj-cs.2472
- Alsaffar, A. M., Nouri-Baygi, M., & Zolbanin, H. M. (2024). Shielding networks: Enhancing intrusion detection with hybrid feature selection and stack ensemble learning. Journal of Big Data, 11(1), 133. https://doi.org/10.1186/s40537-024-00994-7
- Amien, J. Al, Sunanto, S., Rangkuti, M. A.-I., & Soni, S. (2025). Deteksi serangan dalam ekosistem IoT melalui analisis multi-class dengan model XGBoost dan penerapan teknik imbalance ratio pada dataset IoTID20. Jurnal CoSciTech (Computer Science and Information Technology), 6(3), 568–574. https://doi.org/10.37859/coscitech.v6i3.9861
- Bouke, M. A., & Abdullah, A. (2023). An empirical study of pattern leakage impact during data preprocessing on machine learning-based intrusion detection models reliability. Expert Systems with Applications, 230, 120715. https://doi.org/10.1016/j.eswa.2023.120715
Similar Articles
Explore related articles published in this journal.