Bidirectional Encoder Representations from Transformers Fine-Tuning for Sentiment Classification of Cek Bansos Reviews
DOI:
https://doi.org/10.30812/ijecsa.v4i1.4981Keywords:
Cek Application, BERT Sentiment Analysis, Streamlit, User ReviewsAbstract
Social assistance programs are essential government initiatives aimed at supporting underprivileged communities. One such program is facilitated through the Cek Bansos application, which enables users to check their eligibility for social aid. However, user experiences with the application vary, leading to various sentiments in their reviews. Understanding these sentiments is crucial for improving the application’s functionality and user satisfaction. This study focuses on sentiment analysis of user reviews of the Cek Bansos application by leveraging a fine-tuned Indonesian-language Bidirectional Encoder Representations from Transformers (BERT) model. This research aims to evaluate the BERT model's effectiveness in classifying sentiments in user reviews and provide insights that could improve the Cek Bansos application. This research method is the BERT model was fine-tuned using hyperparameters such as a learning rate of 3e-6, batch size of 16, and 9 epochs. The dataset consisted of 8,000 reviews, divided into training (70%), validation (20.1%), and test (9.9%) sets. Review scores were manually categorized, where ratings of 1 to 2 were classified as negative sentiment, 3 as neutral, and 4 to 5 as positive. The results of this research are as follows: the fine-tuned model achieved an accuracy of 77%, with additional evaluation metrics such as precision, recall, and F1 score, demonstrating the model's effectiveness in identifying positive, negative, and neutral sentiments separately. This study concludes that the BERT model provides a reliable method for sentiment classification of user reviews, which could support developers and policymakers in refining the Cek Bansos application to enhance user experience. Additionally, a web-based application developed using Streamlit allows government officials to visualize sentiment trends in real time, improving their understanding of user feedback. Future research could further explore alternative machine learning models and additional linguistic features to improve sentiment classification accuracy and the overall user experience.
References
[1] E. D. Madyatmadja, H. Candra, J. Nathaniel, M. R. Jonathan, and Rudy, “Sentiment Analysis on User Reviews of Threads Applications in Indonesia,” Journal Europeen des Systemes Automatises, vol. 57, no. 4, pp. 1165–1171, 2024, doi: 10.18280/jesa.570423.
[2] K. S. Nugroho, A. Y. Sukmadewa, H. Wuswilahaken Dw, F. A. Bachtiar, and N. Yudistira, “BERT Fine-Tuning for Sentiment Analysis on Indonesian Mobile Apps Reviews,” ACM International Conference Proceeding Series, pp. 258–264, 2021, doi: 10.1145/3479645.3479679.
[3] S. N. Asvia, A. Rahmatulloh, H. Mubarok, A. Setiawan, D. Kusumastuti, and W. Cahyati, “Comparison of Naïve Bayes and Random Forest on Sentiment Analysis of Brand Reputation Provider Live. On,” in 2024 18th International Conference on Telecommunication Systems, Services, and Applications (TSSA), IEEE, Oct. 2024, pp. 1–5. doi: 10.1109/TSSA63730.2024.10863962.
[4] A. Rahmatulloh, R. N. Shofa, I. Darmawan, and Ardiansah, “Sentiment Analysis of Ojek Online User Satisfaction Based on the Naïve Bayes and Net Brand Reputation Method,” in 2021 9th International Conference on Information and Communication Technology (ICoICT), IEEE, Aug. 2021. doi: 10.1109/ICoICT52021.2021.9527466.
[5] F. F. Sabiq, A. Rahmatulloh, I. Darmawan, R. Rizal, R. Gunawan, and E. Haerani, “Performance Comparison of Multinomial and Bernoulli Naïve Bayes Algorithms with Laplace Smoothing Optimization in Fake News Classification,” in 2024 International Conference on Artificial Intelligence, Blockchain, Cloud Computing, and Data Analytics (ICoABCD), IEEE, Aug. 2024, pp. 19–24. doi: 10.1109/ICoABCD63526.2024.10704399.
[6] S. Elmeftahi, M. D. Rakhman, and A. Rahmatulloh, “Comparison of Machine Learning Algorithms in Detecting Contaminants in Drinkable Water,” Innovation in Research of Informatics (INNOVATICS), vol. 6, no. 1, Mar. 2024, doi: 10.37058/innovatics.v6i1.10385.
[7] R. Ardianto, T. Rivanie, Y. Alkhalifi, F. S. Nugraha, and W. Gata, “Sentiment Analysis On E-Sports For Education Curriculum Using Naive Bayes And Support Vector Machine,” Jurnal Ilmu Komputer dan Informasi, vol. 13, no. 2, pp. 109–122, Jul. 2020, doi: 10.21609/jiki.v13i2.885.
[8] Pristiyono, M. Ritonga, M. A. Al Ihsan, A. Anjar, and F. H. Rambe, “Sentiment analysis of COVID-19 vaccine in Indonesia using Naïve Bayes Algorithm,” IOP Conf Ser Mater Sci Eng, vol. 1088, no. 1, p. 012045, Feb. 2021, doi: 10.1088/1757-899X/1088/1/012045.
[9] W. C. Leong, R. O. Kelani, and Z. Ahmad, “Prediction of air pollution index (API) using support vector machine (SVM),” J Environ Chem Eng, vol. 8, no. 3, p. 103208, Jun. 2020, doi: 10.1016/j.jece.2019.103208.
[10] A. Kurani, P. Doshi, A. Vakharia, and M. Shah, “A Comprehensive Comparative Study of Artificial Neural Network (ANN) and Support Vector Machines (SVM) on Stock Forecasting,” Annals of Data Science, vol. 10, no. 1, pp. 183–208, Feb. 2023, doi: 10.1007/s40745-021-00344-x.
[11] D. Mustafa Abdullah and A. Mohsin Abdulazeez, “Machine Learning Applications based on SVM Classification A Review,” Qubahan Academic Journal, vol. 1, no. 2, pp. 81–90, Apr. 2021, doi: 10.48161/qaj.v1n2a50.
[12] A. K. Kalusivalingam, A. Sharma, N. Patel, and V. Singh, “Leveraging BERT and LSTM for Enhanced Natural Language Processing in Clinical Data Analysis,” International Journal of AI and ML, vol. 2, no. 3, Feb. 2021, Accessed: Mar. 28, 2025. [Online]. Available: https://www.cognitivecomputingjournal.com/index.php/IJAIML-V1/article/view/82
[13] N. M. Gardazi, A. Daud, M. K. Malik, A. Bukhari, T. Alsahfi, and B. Alshemaimri, “BERT applications in natural language processing: a review,” Artif Intell Rev, vol. 58, no. 6, p. 166, Mar. 2025, doi: 10.1007/s10462-025-11162-5.
[14] B. K. Mandal, P. Majumder, and B. P. Tewari, “Role of BERT Model for Sequential Text Classification in Biomedical Abstracts,” 2025, pp. 67–82. doi: 10.1007/978-981-97-8627-5_5.
[15] A. S. Talaat, “Sentiment analysis classification system using hybrid BERT models,” J Big Data, vol. 10, no. 1, 2023, doi: 10.1186/s40537-023-00781-w.
[16] R. Rizal, A. Faturahman, A. Impron, I. Darmawan, E. Haerani, and A. Rahmatulloh, “Unveiling the Truth: Detecting Fake News Using SVM and TF-IDF,” in 2025 International Conference on Advancement in Data Science, E-learning and Information System (ICADEIS), IEEE, Feb. 2025, pp. 1–6. doi: 10.1109/ICADEIS65852.2025.10933324.
[17] N. Husin, “Komparasi Algoritma Random Forest, Naïve Bayes, dan Bert Untuk Multi-Class Classification Pada Artikel Cable News Network (CNN),” Jurnal Esensi Infokom : Jurnal Esensi Sistem Informasi dan Sistem Komputer, vol. 7, no. 1, pp. 75–84, 2023, doi: 10.55886/infokom.v7i1.608.
[18] T. Tang, X. Tang, and T. Yuan, “Fine-Tuning BERT for Multi-Label Sentiment Analysis in Unbalanced Code-Switching Text,” IEEE Access, vol. 8, pp. 193248–193256, 2020, doi: 10.1109/ACCESS.2020.3030468.
[19] N. J. Prottasha et al., “Transfer Learning for Sentiment Analysis Using BERT Based Supervised Fine-Tuning,” Sensors, vol. 22, no. 11, pp. 1–19, 2022, doi: 10.3390/s22114157.
[20] Serly Setyani, “Multi Aspect Sentiment Analysis of Mutual Funds Investment App Bibit Using BERT Method,” International Journal on Information and Communication Technology (IJoICT), vol. 9, no. 1, pp. 44–56, 2023, doi: 10.21108/ijoict.v9i1.718.
[21] A. Areshey and H. Mathkour, “Transfer Learning for Sentiment Classification Using Bidirectional Encoder Representations from Transformers (BERT) Model,” Sensors, vol. 23, no. 11, 2023, doi: 10.3390/s23115232.
[22] I. Wickramasinghe and H. Kalutarage, “Naive Bayes: applications, variations and vulnerabilities: a review of literature with code snippets for implementation,” Soft comput, vol. 25, no. 3, pp. 2277–2293, 2021, doi: 10.1007/s00500-020-05297-6.
[23] S. Salnikova and R. Kyrychenko, “Sentiment Analysis Based on the BERT Model: Attitudes Towards Politicians Using Media Data,” Proceedings of the International Conference on Social Science, Psychology and Legal Regulation (SPL 2021), vol. 617, no. Spl, pp. 39–44, 2021, doi: 10.2991/assehr.k.211218.007.
[24] V. A. Flores, P. A. Permatasari, and L. Jasa, “Penerapan Web Scraping Sebagai Media Pencarian dan Menyimpan Artikel Ilmiah Secara Otomatis Berdasarkan Keyword,” Majalah Ilmiah Teknologi Elektro, vol. 19, no. 2, p. 157, 2020, doi: 10.24843/mite.2020.v19i02.p06.
[25] I. G. T. Isa and B. Junedi, “Hyperparameter Tuning Epoch dalam Meningkatkan Akurasi Data Latih dan Data Validasi pada Citra Pengendara,” Prosiding Sains Nasional dan Teknologi, vol. 12, no. 1, p. 231, 2022, doi: 10.36499/psnst.v12i1.6697.
[26] E. Verianto, “Mencegah Overfitting Pada Model Prediksi,” Jurnal Sistem Informasi dan Sistem Komputer, vol. 9, no. 2, pp. 195–204, 2024.
[27] T. Pratama and Suharjito, “IndoXLNet: Pre-Trained Language Model for Bahasa Indonesia,” International Journal of Engineering Trends and Technology, vol. 70, no. 5, pp. 366–380, 2022, doi: 10.14445/22315381/IJETT-V70I5P240.
[28] H. H. Zain, R. M. Awannga, and W. I. Rahayu, “Perbandingan Model Svm, Knn Dan Naïve Bayes Untuk Analisis Sentiment Pada Data Twitter: Studi Kasus Calon Presiden 2024,” JIMPS: Jurnal Ilmiah Mahasiswa Pendidikan Sejarah, vol. 8, no. 3, pp. 2083–2093, 2023.
[29] M. Rizqi, A. Rustiawan, and P. T. Prasetyaningrum, “Analisis Sentimen Terhadap Klinik Natasha Skincare di Yogyakarta Dengan Metode Google Review,” Journal of Information Technology Ampera, vol. 5, no. 1, pp. 2774–2121, 2024, doi: 10.51519/journalita.v5i1.556.
[30] A. Putranto, N. L. Azizah, and A. I. Ratna Ika, “Sistem Prediksi Penyakit Jantung Berbasis Web Menggunakan Metode SVM dan Framework Streamlit,” Jurnal Penerapan Sistem Informasi (Komputer & Manajemen), vol. 4, no. 2, pp. 442–452, 2023.
Downloads
Published
Issue
Section
License
Copyright (c) 2025 Erna Haerani, Alam Rahmatulloh, Souhayla Elmeftahi
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
