A Segmentation-Aware Isolation Forest Framework for Fraud Investigation Prioritization in QRIS Transactions

Ahmad Rijaluddin Nisbil Kamal; Valeriana Lukitosari

doi:10.57152/malcom.v6i3.2832

Authors

Ahmad Rijaluddin Nisbil Kamal Institut Teknologi Sepuluh Nopember
Valeriana Lukitosari Institut Teknologi Sepuluh Nopember

DOI:

https://doi.org/10.57152/malcom.v6i3.2832

Keywords:

Fraud Investigation Prioritization, Isolation Forest, Merchant-Level Behavioral Profiling, QRIS Transactions, Segmentation-Aware Anomaly Detection

Abstract

Fraud detection in Quick Response Code Indonesian Standard (QRIS) transaction environments is challenging because fraud labels are often limited, delayed, or incomplete, making it difficult to apply supervised learning effectively. In addition, high transaction volumes create operational constraints for investigation teams, requiring an efficient mechanism to prioritize high-risk merchants for review. To address this problem, this study proposes a segmentation-aware Isolation Forest framework for fraud investigation prioritization in QRIS transactions. Transaction-level data from a national bank in Indonesia were transformed into merchant-level behavioral representations using 38 engineered features capturing transaction intensity, temporal patterns, nominal concentration, burst behavior, and transactional irregularities. Anomaly scores were then used as ranking signals for prioritizing selective investigation. Model performance was evaluated using proxy-based ranking metrics, capacity simulation investigations, selective alert optimization, transaction-level interpretability, and expert validation. The results show that only 1.03% of merchants were classified as very high anomalies, while the model achieved strong ranking performance with an Area Under the Curve (AUC) of 0.8794, Precision@Top 5% 0.3306, and Lift of 6.5942. Selective alerting reduced investigation workload by up to 80.24%, with higher precision and lower FAR, although at the cost of reduced recall. Expert validation further indicated 94.12% prioritized merchants were operationally relevant.

Downloads

Download data is not yet available.

References

Bank Indonesia, Bank Indonesia: Menavigasi Sistem Pembayaran Nasional di Era Digital. Jakarta: Bank Indonesia, 2019.

Bank Indonesia, “QRIS Jelajah Indonesia 2025 Dorong Digitalisasi dengan Wisata Budaya,” Indonesia, 2025. [Online]. Available: https://www.bi.go.id/id/publikasi/ruang-media/news-release/Pages/sp_2717025.aspx. [Accessed: Oct. 10, 2025].

D. Kumar and S. Singh, “Analyzing the Impact of Machine Learning Algorithms on Risk Management and Fraud Detection in Financial Institution,” Int. J. Res. Publ. Rev., vol. 5, no. 5, pp. 1797–1804, 2024, doi: 10.55248/gengpi.5.0524.1135.

J. de J. Rocha-Salazar, M. J. Segovia-Vargas, and M. del M. Camacho-Miñano, “Money laundering and terrorism financing detection using neural networks and an abnormality indicator,” Expert Syst. Appl., vol. 169, p. 114470, 2021, doi: 10.1016/j.eswa.2020.114470.

A. A. Compagnino et al., “An Introduction to Machine Learning Methods for Fraud Detection,” Appl. Sci., vol. 15, no. 21, pp. 1–32, 2025, doi: 10.3390/app152111787.

M. Abdurrochman et al., “Optimized Machine Learning Model for Credit Card Fraud Detection Using Smote-Tomek and Feature Engineering,” J. Appl. Informatics Comput., vol. 8, no. 2, pp. 580–588, 2024, doi: 10.30871/jaic.v8i2.8732.

A. Abdallah, M. A. Maarof, and A. Zainal, “Fraud detection system?: A survey,” J. Netw. Comput. Appl., vol. 68, pp. 90–113, 2016, doi: 10.1016/j.jnca.2016.04.007.

A. D. Pozzolo, G. Boracchi, O. Caelen, and C. Alippi, “Credit Card Fraud Detection?: A Realistic Modeling and a Novel Learning Strategy,” IEEE Trans. NEURAL NETWORKS Learn. Syst., vol. 29, no. 8, pp. 3784–3797, 2017, doi: 10.1109/TNNLS.2017.2736643.

M. R. Reddy, H. Q. Owaied, M. M. Abdulhasan, P. Karthik, M. Rajkumar, and P. U. Kumar, “Fraud Detection of Online transaction Using Machine Learning,” 2024 Int. Conf. Augment. Reality, Intell. Syst. Ind. Autom. (ARIIA), Manipal, India, 2024, pp. 1–7, 2024, doi: 10.1109/ARIIA63345.2024.11051851.

M. A. Talukder et al., “Machine learning-based network intrusion detection for big and imbalanced data using oversampling, stacking feature embedding and feature extraction,” J. Big Data, vol. 11, no. 1, p. 33, 2024, doi: 10.1186/s40537-024-00886-w.

P. Vanini, S. Rossi, E. Zvizdic, and T. Domenig, “Online payment fraud: from anomaly detection to risk management,” Financ. Innov., vol. 9, no. 1, p. 66, 2023, doi: 10.1186/s40854-023-00470-w.

F. T. Liu, K. M. Ting, and Z. H. Zhou, “Isolation forest,” 2008 Eighth IEEE Int. Conf. Data Mining, Pisa, Italy, pp. 413–422, 2008, doi: 10.1109/ICDM.2008.17.

S. Ounacer, H. A. El Bour, Y. Oubrahim, M. Y. Ghoumari, and M. Azzouazi, “Using Isolation Forest in anomaly detection: The case of credit card transactions,” Period. Eng. Nat. Sci., vol. 6, no. 2, pp. 394–400, 2018, doi: 10.21533/pen.v6i2.533.

H. Abbassi, S. E. L. Mendili, and Y. Gahi, “Digital banking fortification: a real-time isolation forest architecture for detecting online transaction fraud,” Eng. Res. Express, vol. 6, no. 2, p. 25214, 2024, doi: 10.1088/2631-8695/ad4958.

M. S. Kareem and L. A. N. Muhammed, “Anomaly Detection in Streaming Data using Isolation Forest,” 2024 Seventh Int. Women Data Sci. Conf. Prince Sultan Univ. (WiDS PSU), Riyadh, Saudi Arab., pp. 223–228, 2024, doi: 10.1109/WiDS-PSU61003.2024.00052.

J. Nanduri, Y. Jia, A. Oka, J. Beaver, and Y. Liu, “Microsoft Uses Machine Learning and Optimization to Reduce E-Commerce Fraud,” INFORMS J. Appl. Anal., vol. 50, no. 1, pp. 64–79, 2020, doi: 10.1287/inte.2019.1017.

M. Kuhn and K. Johnson, Applied Predictive Modeling, 1st ed. New York: Springer, 2013. doi: 10.1007/978-1-4614-6849-3.

Y. Chabchoub, M. U. Togbe, A. Boly, and R. Chiky, “An In-Depth Study and Improvement of Isolation Forest,” IEEE Access, vol. 10, pp. 10219–10237, 2022, doi: 10.1109/ACCESS.2022.3144425.

H. Xu, G. Pang, Y. Wang, and Y. Wang, “Deep Isolation Forest for Anomaly Detection,” IEEE Trans. Knowl. Data Eng., vol. 35, no. 12, pp. 12591–12604, 2023, doi: 10.1109/TKDE.2023.3270293.

C. C. Aggarwal, Outlier Analysis. Cham: Springer International Publishing, 2017. doi: 10.1007/978-3-319-47578-3.