Evaluasi Machine Learning untuk Prediksi Pembatalan Hotel dengan Threshold Adjustment dan Cost-Based Evaluation: Machine Learning Evaluation for Hotel Cancellation Prediction with Threshold Adjustment and Cost-Based Evaluation

Efraim William Solang; Franco Xander Adu; Agus Dharma; Nyoman Gunantara

doi:10.57152/malcom.v6i1.2466

Authors

Efraim William Solang Universitas Udayana
Franco Xander Adu Universitas Udayana
Agus Dharma Universitas Udayana
Nyoman Gunantara Universitas Udayana

DOI:

https://doi.org/10.57152/malcom.v6i1.2466

Keywords:

Cost-Based Evaluation, Hotel Booking Demand, Machine Learning, Pembatalan Pemesanan Hotel

Abstract

Pembatalan pemesanan hotel merupakan permasalahan krusial yang berdampak langsung pada pendapatan dan perencanaan operasional. Penelitian ini mengevaluasi penerapan threshold adjustment dan cost-based evaluation untuk meningkatkan kualitas pengambilan keputusan bisnis. Penelitian ini melibatkan perbandingan beberapa jenis model machine learning menggunakan dataset hotel booking demand. Kinerja model dinilai menggunakan metrik F0.5-Score, precision, ROC AUC, dan pendekatan cost-based evaluation berbasis net revenue. Hasil penelitian menunjukkan bahwa Random Forest memberikan kinerja terbaik dengan F0.5-Score 0.8279, precision 0.878 dan ROC AUC 0.9165. Model lain seperti Logistic Regression (baseline) dengan F0.5-Score 0.7816, XGBoost dengan F-.5-Score 0.8108 dan ANN dengan F0.5-Score 0.8091 menunjukan performa lebih relatif lebih rendah, mengindikasikan bahwa dataset ini lebih cocok menggunakan pendekatan ensamble learning. Temuan penting mengungkapkan bahwa penyesuaian threshold berdasarkan F0.5-Score tidak selalu menghasilkan keuntungan ekonomi maksimum. Penggunaan threshold (0.52) terbukti menghasilkan nilai net revenue lebih tinggi dibandingkan threshold optimal berbasis F0.5-Score. Pendekatan ini diharapkan dapat meningkatkan kualitas pengambilan keputusan bisnis bagi manajer hotel dalam pengelolaan risiko finansial.

Downloads

Download data is not yet available.

References

S. Ivanov and V. Zhechev, “TOURISM Review Hotel revenue Management - A Critical Literature Review,” Jan. 2012.

B. M. Noone and C. H. Lee, “Hotel overbooking: The effect of overcompensation on customers’ reactions to denied service,” Journal of Hospitality and Tourism Research, vol. 35, no. 3, pp. 334–357, Aug. 2011, doi: 10.1177/1096348010382238.

Z. Kenesei and Z. Bali, “Overcompensation as a service recovery strategy: the financial aspect of customers’ extra effort,” Service Business, vol. 14, no. 2, pp. 187–216, Jun. 2020, doi: 10.1007/s11628-020-00413-w.

N. Antonio, A. de Almeida, and L. Nunes, “Hotel booking demand datasets,” Data Brief, vol. 22, pp. 41–49, Feb. 2019, doi: 10.1016/j.dib.2018.11.126.

T. Y. Alkan, “A Comparative Study of Machine Learning and Deep Learning Approaches For Hotel Booking Cancellation Prediction,” International Journal Of Scientific Research In Engineering & Technology, pp. 11–18, May 2025, doi: 10.59256/ijsreat.20250503002.

A. Onan and S. KorukoGlu, “A feature selection model based on genetic rank aggregation for text sentiment classification,” J Inf Sci, vol. 43, no. 1, pp. 25–38, Feb. 2017, doi: 10.1177/0165551515613226.

L. Grinsztajn, E. Oyallon, and G. Varoquaux, “Why do tree-based models still outperform deep learning on tabular data?,” Jul. 2022, [Online]. Available: http://arxiv.org/abs/2207.08815

Z. A. Andriawan et al., “Prediction of Hotel Booking Cancellation using CRISP-DM,” in ICICoS 2020 - Proceeding: 4th International Conference on Informatics and Computational Sciences, Institute of Electrical and Electronics Engineers Inc., Nov. 2020. doi: 10.1109/ICICoS51170.2020.9299011.

A. Onan, “Consensus Clustering-Based Undersampling Approach to Imbalanced Learning,” Sci Program, vol. 2019, 2019, doi: 10.1155/2019/5901087.

S. Domínguez-Almendros, N. Benítez-Parejo, and A. R. Gonzalez-Ramirez, “Logistic regression models,” Allergol Immunopathol (Madr), vol. 39, no. 5, pp. 295–305, Sep. 2011, doi: 10.1016/j.aller.2011.05.002.

N. Antonio, A. De Almeida, and L. Nunes, “An automated machine learning based decision support system to predict hotel booking cancellations,” Data Sci J, vol. 18, no. 1, 2019, doi: 10.5334/dsj-2019-032.

M. Sagala, “Algorithm Modified K-Nearest Neighbor (M-KNN) for Classification of Attention Deficit Hyperactive Disorder (ADHD) in Children,” 2019. [Online]. Available: http://login.seaninstitute.org/index.php/Login?11Journalhomepage:http://login.seaninstitute.org/index.php/Login

F. Akbar, H. Wira Saputra, A. Karel Maulaya, and M. Fikri Hidayat, “Implementasi Algoritma Decision Tree C4.5 dan Support Vector Regression untuk Prediksi Penyakit Stroke,” vol. 2, pp. 61–67, Oct. 2022.

C. Bentéjac, A. Csörgö, and G. Martínez-Muñoz, “A Comparative Analysis of XGBoost,” Nov. 2019. doi: 10.48550/arXiv.1911.01914.

P. Florek and A. Zagda?ski, “Benchmarking state-of-the-art gradient boosting algorithms for classification,” May 2023, [Online]. Available: http://arxiv.org/abs/2305.17094

J. G. Cabello, “Mathematical Neural Networks,” Axioms, vol. 11, no. 2, Feb. 2022, doi: 10.3390/axioms11020080.

M. Mujahid et al., “Data Oversampling and Imbalanced Datasets: An Investigation of Performance For Machine Learning and Feature Engineering,” J Big Data, vol. 11, no. 1, Dec. 2024, doi: 10.1186/s40537-024-00943-4.

P. Peykani, M. Peymany Foroushany, C. Tanasescu, M. Sargolzaei, and H. Kamyabfar, “Evaluation of Cost-Sensitive Learning Models in Forecasting Business Failure of Capital Market Firms,” Mathematics, vol. 13, no. 3, Feb. 2025, doi: 10.3390/math13030368.

N. Phumchusri and P. Maneesophon, “Optimal overbooking decision for hotel rooms revenue management,” Journal of Hospitality and Tourism Technology, vol. 5, no. 3, pp. 261–277, Oct. 2014, doi: 10.1108/JHTT-03-2014-0006.

M. J. Ariza-Garzón, J. Arroyo, M. J. Segovia-Vargas, and A. Caparrini, “Profit-sensitive machine learning classification with explanations in credit risk: The case of small businesses in peer-to-peer lending,” Electron Commer Res Appl, vol. 67, Sep. 2024, doi: 10.1016/j.elerap.2024.101428.

C. Yaiprasert and A. N. Hidayanto, “AI-powered ensemble machine learning to optimize cost strategies in logistics business,” International Journal of Information Management Data Insights, vol. 4, no. 1, Apr. 2024, doi: 10.1016/j.jjimei.2023.100209.

L. Cohen, Y. Mansour, S. Moran, and H. Shao, “Probably Approximately Precision and Recall Learning,” Oct. 2025, [Online]. Available: http://arxiv.org/abs/2411.13029

S. Beddar-Wiesing, A. Moallemy-Oureh, M. Kempkes, and J. M. Thomas, “Absolute Evaluation Measures for Machine Learning: A Survey,” Jul. 2025, [Online]. Available: http://arxiv.org/abs/2507.03392

O. Rainio, J. Teuho, and R. Klén, “Evaluation metrics and statistical tests for machine learning,” Sci Rep, vol. 14, no. 1, Dec. 2024, doi: 10.1038/s41598-024-56706-x.

B. Hutchinson, N. Rostamzadeh, C. Greer, K. Heller, and V. Prabhakaran, “Evaluation Gaps in Machine Learning Practice,” in ACM International Conference Proceeding Series, Association for Computing Machinery, Jun. 2022, pp. 1859–1876. doi: 10.1145/3531146.3533233.

M. Bhagat and B. Bakariya, “A Comprehensive Review of Cross-Validation Techniques in Machine Learning,” 2025.

P. Probst, M. Wright, and A.-L. Boulesteix, “Hyperparameters and Tuning Strategies for Random Forest,” Feb. 2019, doi: 10.1002/widm.1301.

V. Komisarenko and M. Kull, “Cost-sensitive classification with cost uncertainty: do we need surrogate losses?,” Mach Learn, vol. 114, no. 6, Jun. 2025, doi: 10.1007/s10994-024-06634-8.

N. Lawrance, M.-A. Guerry, and G. Petrides, “Cost-Sensitive Stacking: an Empirical Evaluation,” Jan. 2023, [Online]. Available: http://arxiv.org/abs/2301.01748