Sentiment Classification of Football Supporters Using NusaBERT Embeddings, BiLSTM, and BiGRU Methods
Authors
-
Muhammad Guntara
Universitas Bumigora, Mataram, Indonesia
-
I Gusti Ayu Diah Gita Kartika Santi
Universitas Bumigora, Mataram, Indonesia
-
Hairani Hairani
Universitas Bumigora, Mataram, Indonesia
-
Lalu Zazuli Azhar Mardedi
Universitas Bumigora, Mataram, Indonesia
DOI:
https://doi.org/10.30812/ijecsa.v5i1.6151Keywords:
BiLSTM, BiGRU, NusaBERT, Sentiment Analysis, SMOTEAbstract
Class imbalance and the use of non-standard language in football supporters’ opinions on social media constitute major obstacles to producing accurate sentiment classification for evaluating federation performance. This study aims to identify the most effective bidirectional recurrent architecture for capturing public opinion after applying data balancing techniques. Using a primary dataset of 1,039 instances (604 positive and 435 negative samples), the proposed method integrates a pre-trained NusaBERT model with hybrid Bidirectional Long Short-Term Memory (BiLSTM) and Bidirectional Gated Recurrent Unit (BiGRU) layers. To address data imbalance, the Synthetic Minority Over-sampling Technique (SMOTE) is applied to the training data, with dataset partitioning using a stratified split ratio of 70:30. The results indicate that the NusaBERT-BiLSTM model achieves the best performance, with a testing accuracy of 70.83% and an F1-score of 0.6990, outperforming the BiGRU variant, which attains an accuracy of only 64.74%. Furthermore, NusaBERT-BiLSTM demonstrates greater reliability in detecting negative sentiment, achieving a recall value of 0.6336 compared to 0.4504 for BiGRU. In conclusion, combining NusaBERT's semantic strength with SMOTE-based balancing and BiLSTM layers significantly enhances the model’s sensitivity to minority opinions without causing data leakage. This study contributes a more objective classification model for national team management to accurately map public criticism and aspirations on social media.
Downloads
References
[1] A. Riska, E. Saputri, F. Azzahra, dan M. R. Aditya, “Analisis respon netizen terhadap pemutusan kontrak coach shin tae-yong di postingan instagram @ timnasindonesia,” J. Komun. Univ. Garut Has. Pemikir. dan Penelit., vol. 11, no. 1, hal. 71–89, 2025, doi: doi.org/10.52434/jk.v11i1.42101.
[2] A. Ahmad Tenriliweng Hasan dan N. Fridayanti, “Social Media as a Catalyst for Supporter Conflicts and Riots in Indonesia,” Komunikator, vol. 17, no. 1, hal. 95–109, 2025, doi: https://doi.org/jkm.v17i1.26044.
[3] S. Nurika Indriyana dan F. Pangesti, “Kritik dan Sarkasme Pada Kinerja Gubernur Anies Baswedan di Twitter : Kajian Sosolinguistik ( Criticism and Sarcasm on the Performance of Governor Anies Baswedan on Twitter : A Sosolinguistic Study ),” Indones. Lang. Educ. Lit., vol. 8, no. 1, hal. 100–113, 2022, doi: 10.24235/ileal.v8i1.10113.
[4] D. Wulan, Y. Rahayu, K. Umam, dan M. R. Handayani, “Performance of Machine Learning Algorithms on Imbalanced Sentiment Datasets Without Balancing Techniques,” J. Appl. Informatics Comput., vol. 9, no. 3, hal. 998–1005, 2025, doi: https://doi.org/10.30871/jaic.v9i3.9584.
[5] D. Ferdiana Merpatika dan A. Yakobus CHandra, “ANALISIS SENTIMEN PUBLIK DI MEDIA SOSIAL TERHADAP KASUS DUGAAN KORUPSI IMPOR MINYAK PERTAMINA MENGGUNAKAN XGBOOST,” RABIT J. Teknol. dan Sist. Inf. Univrab, vol. 10, no. 2, hal. 576–593, 2025, doi: https://doi.org/ 10.36341/rabit.v10i2.6307.
[6] W. Alfiyani, D. A. Fatah, dan F. Irhamni, “PENERAPAN ALGORITMA NAÏVE BAYES UNTUK ANALISIS SENTIMEN PADA MEDIA SOSIAL X TERHADAP PERFORMA TIM NASIONAL SEPAK BOLA INDONESIA DI ERA KEPEMIMPINAN SHIN TAE-YONG,” JATI (Jurnal Mhs. Tek. Inform., vol. 9, no. 3, hal. 3969–3977, 2025, doi: https://doi.org/10.36040/jati.v9i3.13451.
[7] N. Modami, E. Eleazar Reva Manopo, D. R. Enditama, dan A. T. Ayunda, “Analisis Sentimen Komentar YouTube terhadap Rumor Peluncuran iPhone 17 Menggunakan Web Scraping dan Studi Komparatif Algoritma Klasifikasi,” JOISIE (Journal Inf. Syst. Informatics Eng., vol. 9, no. 2, hal. 403–411, 2025, doi: https://doi.org/10.35145/joisie.v9i2.5684.
[8] J. Saputra, L. Maryani, D. Wulandari, W. Eka, P. T. Informatika, dan P. T. Komputer, “Analisis Performa Naive Bayes dan SVM terhadap Sentimen Teks Media Sosial dengan Word2Vec dan SMOTE,” INSTEK, vol. 10, no. 1, hal. 143–155, 2025, doi: https://doi.org/10.24252/instek.v10i1.54889.
[9] Rizal, Junaedy, dan B. Herlinah, “IMPLEMENTASI METODE LONG SHORT-TERM MEMORY (LSTM) UNTUK ANALISIS SENTIMEN KENDARAAN LISTRIK PADA TWITTER (X),” J. Teknol. Dan Komput., vol. 5, no. 1, hal. 566–582, 2025, doi: https://doi.org/10.56923/jtek.v5i01.217.
[10] S. M. Natzir dan H. Jatiprasetya, “Prediksi harga cryptocurrency xlm menggunakan metode deep learning lstm dan gru,” HOAQ J. Teknol. Inf., vol. 16, no. c, hal. 49–58, 2025, doi: https://doi.org/10.52972/hoaq.vol16no1.
[11] M. S. Sambo, S. Trihandaru, D. B. Nugroho, dan H. A. Parhusip, “Comparative Analysis of LSTM and Bi-LSTM for Classifying Indonesian New Translation Bible Texts Using Word2Vec Embedding,” CommIT (Communication Inf. Technol. J., vol. 19, no. 2, hal. 183–202, 2025, doi: https://doi.org/10.21512/commit.v19i2.12015.
[12] N. Sofa, F. S. Utomo, dan R. E. Saputro, “Eksplorasi Model Hybrid Transformer-Latent Semantic Analysis ( LSA ) Untuk Pemahaman Konteks Teks Berita Berbahasa Indonesia,” J. Pendidik. dan Teknol. Indones., vol. 5, no. 5, hal. 1239–1252, 2025, doi: https://doi.org/10.52436/1.jpti.662.
[13] M. Amien, G. F. Gunawan, I. Artikel, dan K. Kunci, “BERT dan Bahasa Indonesia : Studi tentang Efektivitas Model NLP Berbasis Transformer,” ELANG J. ofInterdisciplinary Res., vol. 1, no. 2, hal. 132–140, 2024, doi: https://doi.org/10.32664/elang.v1i02.
[14] V. D. Setiawan, D. U. Iswavigra, dan E. Anggiratih, “Implementation of IndoBERT for Sentiment Analysis of the Constitutional Court ’ s Decision Regarding the Minimum Age of Vice Presidential Candidates,” Sci. J. Informatics, vol. 12, no. 3, hal. 397–406, 2025, doi: 10.15294/sji.v12i3.26360.
[15] H. Khataei, M. Farhad, S. Gharehchopogh, K. Majidzadeh, dan A. Babazadeh, “A Hybrid Model Based on Convolutional Neural Network and Long Short-Term Memory for Multi-label Text Classification,” Neural Process. Lett., vol. 56, no. 2, hal. 1–31, 2024, doi: 10.1007/s11063-024-11500-8.
[16] D. Tiwari, B. S. Bhati, B. Nagpal, N. Alturki, dan L. Bayosenge, “Attention-augmented hybrid CNN-LSTM model for social media sentiment analysis in cryptocurrency investment decision-making,” Sci. Rep., vol. 15, no. 33, hal. 1–27, 2025, doi: https://doi.org/10.1038/s41598-025-18245-x 1.
[17] M. Faruqziddan, E. Herdika, S. Aulia, S. D. Azzahra, dan P. B. Utomo, “Perbandingan Skenario Balancing Oversampling dan Undersampling dalam Klasifikasi Resiko Kambuh Kanker Tiroid menggunakan Algoritma SVM Linear,” JAMI J. Ahli Muda Indones., vol. 5, no. 2, hal. 172–182, 2024, doi: 10.46510/jami.v5i2.320.
[18] T. D. Purnomo dan J. Sutopo, “COMPARISON OF PRE-TRAINED BERT-BASED TRANSFORMER MODELS FOR REGIONAL LANGUAGE TEXT SENTIMENT ANALYSIS IN INDONESIA,” IJST, vol. 3, no. 3, hal. 11–21, 2024, doi: doi.org/10.56127/ijst.v3i3.1739.
[19] M. A. Talukder, M. Khalid, dan N. Sultana, “A hybrid machine learning model for intrusion detection in wireless sensor networks leveraging data balancing and dimensionality reduction,” Sci. Rep., vol. 15, no. 1, hal. 1–23, 2025, doi: 10.1038/s41598-025-87028-1.
[20] G. S.Uday Kumar, K. A.Armanulla, K. A.Umar Khaleefa, dan G. Blessy, “International Journal of Innovative Research in Science Engineering and Technology ( IJIRSET ) Understanding Emotions through NLP,” Int. J. Innov. Res. Sci. Eng. Technol., vol. 14, no. 4, hal. 2025, 2025, doi: 10.15680/IJIRSET.2025.1404507.
[21] A. S. Talaat, “Hybrid Deep Learning Models for Text Classification : Performance Evaluation of TriDistilBERT and BiGRU Architectures,” J. Comput. Sci. Res., vol. 21, no. 9, hal. 1983–1992, 2025, doi: 10.3844/jcssp.2025.1983.1992.
[22] S. E. Safitri, W. D. Yuniarti, M. R. Handayani, dan K. Umam, “User Opinion Mining on the Maxim Application Reviews Using BERT-Base Multilingual Uncased,” J. SISFOKOM (Sistem Inf. dan Komputer), vol. 14, no. 3, hal. 365–372, 2025, doi: 10.32736/sisfokom.v14i3.2391.
[23] A. S. Talaat, “Sentiment analysis classification system using hybrid BERT models,” J. Big Data, vol. 10, no. 1, hal. 110, 2023, doi: 10.1186/s40537-023-00781-w.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Muhammad Guntara, I Gusti Ayu Diah Gita Kartika Santi, Hairani Hairani
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
