Klasifikasi Hate Speech dan Offensive Language Menggunakan Hybrid RoBERTa dan XGBoost dengan Optimasi Hyperparameter
Hate Speech and Offensive Language Classification Using Hybrid RoBERTa and XGBoost with Hyperparameter Optimization
DOI:
https://doi.org/10.57152/malcom.v6i3.2834Keywords:
HASOC, Hate Speech, Optimasi Hyperparameter, RoBERTa, Ujaran Kebencian, XGBoostAbstract
Ujaran kebencian dan bahasa ofensif di media sosial merupakan masalah serius yang membutuhkan deteksi otomatis yang akurat. Penelitian ini mengusulkan pendekatan hybrid yang menggabungkan model Robustly Optimised BERT Approach (RoBERTa) berbasis Twitter (cardiffnlp/twitter?roberta?base?offensive) sebagai ekstraktor fitur dengan algoritma XGBoost untuk klasifikasi pada dataset Hate Speech and Offensive Content Identification (HASOC) 2021 berbahasa Inggris. Kebaruan penelitian ini terletak pada integrasi optimasi hyperparameter Optuna, Multi?Seed Ensemble, Division Calibration, dan oversampling pada ruang fitur embedding yang belum pernah diterapkan secara bersamaan dalam satu pipeline pada HASOC 2021. Dua tugas diselesaikan : Task 1A adalah klasifikasi biner untuk membedakan konten Hate and Offensive (HOF) dari yang tidak (NOT). Task 1B adalah klasifikasi multi-kelas yang membagi tweet menjadi empat kategori: (ujaran kebencian terhadap kelompok) HATE, bahasa ofensif terhadap individu (OFFN), kata kasar tanpa target spesifik (PRFN), dan konten aman (NONE). Ketidakseimbangan kelas ditangani dengan oversampling pada ruang fitur, dan optimasi hyperparameter dilakukan untuk meningkatkan performa. Hasil evaluasi pada data uji menunjukkan Macro F1?Score sebesar 0,8083 untuk Task 1A dan 0,6541 untuk Task 1B. Perbandingan dengan papan peringkat HASOC 2021 menunjukkan bahwa skor tersebut sebanding dengan tim peringkat 5 (Task 1A) dan peringkat 3 (Task 1B) yang berbasis BERT murni.
Downloads
References
A. Guterres, “Uncontrolled Social Media Networks Amplify The Worst Human Impulses, Allowing Hate Speech To Spread Like Wildfire And Ignite Violence,” Dawn. Accessed: Apr. 17, 2026. [Online]. Available: https://www.dawn.com/news/1874996
X Corp., “Global Transparency Report H2 2024.” Accessed: Jun. 13, 2026. [Online]. Available: https://transparency.x.com/content/dam/transparency-twitter/2025/x-global-transparency-report_h2_2024.pdf
X Corp., “Global Transparency Report H1 2024.” Accessed: Jun. 13, 2026. [Online]. Available: https://transparency.x.com/content/dam/transparency-twitter/2024/x-global-transparency-report-h1.pdf
P. Madriaza et al., “Exposure to hate in online and traditional media: A systematic review and meta-analysis of the impact of this exposure on individuals and communities,” Mar. 01, 2025, John Wiley and Sons Inc. doi: 10.1002/cl2.70018.
University of Oxford, “Majority support moderation on social media platforms, global survey shows,” University of Oxford. Accessed: Apr. 17, 2026. [Online]. Available: https://www.ox.ac.uk/news/2025-02-12-majority-support-moderation-social-media-platforms-global-survey-shows
Republik Indonesia, Undang-Undang Republik Indonesia Nomor 11 Tahun 2008 tentang Informasi dan Transaksi Elektronik sebagaimana telah diubah dengan Undang-Undang Nomor 19 Tahun 2016. Indonesia: Lembaran Negara RI Tahun 2016 No. 251, 2016.
T. Mandl et al., “Overview of the HASOC track at FIRE 2019: Hate Speech and Offensive Content Identification in Indo-European Languages.” [Online]. Available: https://doi.org/10.1145/3368567.3368584
T. Mandl, S. Modha, G. K. Shahi, and others, “Overview of the HASOC Track at FIRE 2020: Hate Speech and Offensive Content Identification in Indo-European Languages,” in CEUR Workshop Proc. (FIRE 2020), 2020. doi: 10.1145/3441501.3441517.
T. Mandl, G. K. Shahi, M. Zampieri, and others, “Overview of the HASOC Subtrack at FIRE 2021: Hate Speech and Offensive Content Identification in English and Indo-Aryan Languages,” in FIRE 2021 (CEUR Workshop Proceedings), 2021. doi: 10.48550/arXiv.2112.09301.
R. Kumar, V. Gupta, and R. Pamula, “Hate Speech and Offensive Content Identification in English Tweets,” 2021. [Online]. Available: https://ceur-ws.org/Vol-3159/T1-46.pdf
M. Mozafari, R. Farahbakhsh, and N. Crespi, “Hate Speech Detection and Racial Bias Mitigation in Social Media Based on BERT Model,” PLoS One, vol. 15, no. 8, p. e0237861, 2020, doi: 10.1371/journal.pone.0237861.
R. Sivanaiah, A. D. S, S. M. Rajendram, and T. T. Mirnalinee, “TECHSSN at SemEval-2020 Task 12: Offensive Language Detection Using BERT Embeddings,” in Proceedings of the 14th International Workshop on Semantic Evaluation (SemEval-2020), 2020. doi: 10.18653/v1/2020.semeval-1.291.
M. Almaliki, A. M. Almars, I. Gad, and E. S. Atlam, “ABMM: Arabic BERT-Mini Model for Hate-Speech Detection on Social Media,” Electronics (Switzerland), vol. 12, no. 4, Feb. 2023, doi: 10.3390/electronics12041048.
J. Salminen, M. Hopf, S. A. Chowdhury, S. gyo Jung, H. Almerekhi, and B. J. Jansen, “Developing an online hate classifier for multiple social media platforms,” Human-centric Computing and Information Sciences 2020 10:1, vol. 10, no. 1, pp. 1-, Jan. 2020, doi: 10.1186/S13673-019-0205-6.
U. Iftikhar, S. F. Ali, G. Mustafa, N. Bahar, and K. Ishaq, “Beyond words: a hybrid transformer-ensemble approach for detecting hate speech and offensive language on social media,” PeerJ Comput. Sci., vol. 11, 2025, doi: 10.7717/peerj-cs.3214.
W. Ashiq et al., “Roman urdu hate speech detection using hybrid machine learning models and hyperparameter optimization,” Sci. Rep., vol. 14, no. 1, Dec. 2024, doi: 10.1038/s41598-024-79106-7.
CardiffNLP, “twitter-roberta-base-offensive.” Accessed: Apr. 16, 2026. [Online]. Available: https://huggingface.co/cardiffnlp/twitter-roberta-base-offensive
F. Barbieri, J. Camacho-Collados, L. Neves, and L. Espinosa-Anke, “TweetEval: Unified Benchmark and Comparative Evaluation for Tweet Classification,” Oct. 2020, doi: 10.48550/arXiv.2010.12421.
Y. Liu et al., “RoBERTa: A Robustly Optimized BERT Pretraining Approach,” Jul. 2019, doi: 10.48550/arXiv.1907.11692.
S. Alsaidi, M. Vincent, O. Boyer, N. Garcelon, M. Couceiro, and A. Coulet, “Comparing representations of long clinical texts for the task of patient note-identification,” Mar. 2025, doi: 10.18653/v1/2025.cl4health-1.9.
B. Li et al., “Equalized Focal Loss for Dense Long-Tailed Object Detection,” Jun. 2022, doi: 10.1109/CVPR52688.2022.00686.
T. Chen and C. Guestrin, “XGBoost: A scalable tree boosting system,” in Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Association for Computing Machinery, Aug. 2016, pp. 785–794. doi: 10.1145/2939672.2939785.
T. Akiba, S. Sano, T. Yanase, T. Ohta, and M. Koyama, “Optuna: A Next-generation Hyperparameter Optimization Framework,” Jul. 2019, doi: 10.1145/3292500.3330701.
M. Sipper, “Neural Networks with À La Carte Selection of Activation Functions,” SN Comput. Sci., vol. 2, no. 6, Nov. 2021, doi: 10.1007/s42979-021-00885-1.
K. Mnassri, P. Rajapaksha, R. Farahbakhsh, and N. Crespi, “BERT-based Ensemble Approaches for Hate Speech Detection,” in Proceedings - IEEE Global Communications Conference, GLOBECOM, Institute of Electrical and Electronics Engineers Inc., 2022, pp. 4649–4654. doi: 10.1109/GLOBECOM48099.2022.10001325.
C. Esposito, G. A. Landrum, N. Schneider, N. Stiefl, and S. Riniker, “GHOST: Adjusting the Decision Threshold to Handle Imbalanced Data in Machine Learning,” Jun. 28, 2021, American Chemical Society. doi: 10.1021/acs.jcim.1c00160.
M. Abdelhamid and A. Desai, “Balancing the Scales: A Comprehensive Study on Tackling Class Imbalance in Binary Classification,” Sep. 2024, doi: 10.48550/arXiv.2409.19751.
N. Bölücü and P. Canbay, “Hate Speech and Offensive Content Identification with Graph Convolutional Networks,” 2021. [Online]. Available: https://ceur-ws.org/Vol-3159/T1-4.pdf
A. Glazkova, M. Kadantsev, and M. Glazkov, “Fine-tuning of Pre-trained Transformers for Hate, Offensive, and Profane Content Detection in English and Marathi,” Oct. 2021, doi: 10.48550/arXiv.2110.12687.
A. Mitra and P. Sankhala, “Multilingual Hate Speech and Offensive Content Detection using Modified Cross-entropy Loss,” Feb. 2022, doi: 10.48550/arXiv.2202.02635.
S. Agustian, R. Saputra, and A. Fadhilah, “‘Feature Selection’ With Pretrained-Bert For Hate Speech And Offensive Content Identification In English And Hindi Languages,” 2021. doi: 10.13140/RG.2.2.27831.60324.
M. Bhatia et al., “One to rule them all: Towards Joint Indic Language Hate Speech Detection,” Sep. 2021, doi: 10.48550/arXiv.2109.13711.
S. Chanda, S. Ujjwal, S. Das, and S. Pal, “Fine-tuning Pre-Trained Transformer based model for Hate Speech and Offensive Content Identification in English, Indo-Aryan and Code-Mixed (English-Hindi) languages,” 2021. [Online]. Available: https://ceur-ws.org/Vol-3159/T1-44.pdf
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Najwa Jauhari, Surya Agustian, Fadhilah Syafria, Muhammad Affandes
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Copyright © by Author; Published by Institut Riset dan Publikasi Indonesia (IRPI)
This Indonesian Journal of Machine Learning and Computer Science is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
