Sentiment Analysis and Topic Modeling of Kitabisa Applications using Support Vector Machine (SVM) and Smote-Tomek Links Methods
Keywords:
Sentiment Analysis, Support Vector Machine, SMOTE-Tomek Links, Latent Dirichlet Allocation
Abstract
Kitabisa is an Indonesian application that functions to raise funds online. Users can easily support various types of campaigns and donate funds to various social causes through the app. User reviews of the application are very diverse, and it is not sure whether user reviews of the application tend to be positive, neutral, or negative. This research aimed to analyze the sentiment of the Kitabisa application by modeling topics using Latent Dirichlet Allocation (LDA) and classifying user reviews using a Support Vector Machine (SVM). The scrapped dataset showed imbalanced dataset problems, so the SMOTE-Tomek Links oversampling technique was proposed. The results of this study show that using LDA produces five topics often discussed in 750 reviews. Then, the performance of SVM without using SMOTE-Tomek Links was 72% accuracy, 76% precision, 72% recall, and 64% f1 score. Meanwhile, using SMOTE-Tomek Links could significantly improve the performance, namely 98% accuracy, 98% precision, 98% recall, and 98% f1 score. Based on this research, the application of SVM achieved high performance for user sentiment classification, especially when the dataset was in a balanced state. Therefore, the SMOTE-Tomek Links oversampling technique is recommended for dealing with unbalanced sentiment datasets.
References
[1] I. Irwansyah and N. Hutami, “Pemanfaatan Aplikasi Mobile Kitabisa Dalam Pelaksanaan Crowdfunding Di Indonesia.,” J. Komun., vol. 13, no. 2, pp. 183–194, 2019, doi: https://doi.org/10.21107/ilkom.v13i2.5357.
[2] F. Fitriana, E. Utami, and H. Al Fatta, “Analisis Sentimen Opini Terhadap Vaksin Covid - 19 pada Media Sosial Twitter Menggunakan Support Vector Machine dan Naive Bayes,” J. Komtika (Komputasi dan Inform., vol. 5, no. 1, pp. 19–25, Jul. 2021, doi: 10.31603/komtika.v5i1.5185.
[3] P. T. Putra, A. Anggrawan, and H. Hairani, “Comparison of Machine Learning Methods for Classifying User Satisfaction Opinions of the PeduliLindungi Application,” Matrik J. Manajemen, Tek. Inform. dan Rekayasa Komput., vol. 22, no. 3, pp. 431–440, 2023, doi: 10.30812/matrik.v22i3.2860.
[4] H. Hairani, A. Anggrawan, A. I. Wathan, K. A. Latif, K. Marzuki, and M. Zulfikri, “The Abstract of Thesis Classifier by Using Naive Bayes Method,” in 2021 International Conference on Software Engineering & Computer Systems and 4th International Conference on Computational Science and Information Management (ICSECS-ICOCSIM), 2021, pp. 312–315. doi: 10.1109/ICSECS52883.2021.00063.
[5] J. W. Fernanda, “Pemodelan Persepsi Pembelajaran Online Menggunakan Latent Dirichlet Allocation,” J. Stat. Univ. Muhammadiyah Semarang, vol. 9, no. 2, pp. 79–85, 2021, doi: 10.26714/jsunimus.9.2.2021.79-85.
[6] M. F. Madjid, D. E. Ratnawati, and B. Rahayudi, “Sentiment Analysis on App Reviews Using Support Vector Machine and Naïve Bayes Classification,” Sink. J. dan Penelit. Tek. Inform., vol. 8, no. 1, pp. 556–562, 2023, [Online]. Available: https://jurnal.polgan.ac.id/index.php/sinkron/article/view/12161
[7] G. K. Locarso, “Analisis Sentimen Review Aplikasi Pedulilindungi Pada Google Play Store Menggunakan NBC,” J. Tek. Inform. Kaputama, vol. 6, no. 2, pp. 353–361, 2022.
[8] S. Rahayu, Y. MZ, J. E. Bororing, and R. Hadiyat, “Implementasi Metode K-Nearest Neighbor (K-NN) untuk Analisis Sentimen Kepuasan Pengguna Aplikasi Teknologi Finansial FLIP,” Edumatic J. Pendidik. Inform., vol. 6, no. 1, pp. 98–106, Jun. 2022, doi: 10.29408/edumatic.v6i1.5433.
[9] O. I. Gifari, M. Adha, F. Freddy, and F. F. S. Durrand, “Analisis Sentimen Review Film Menggunakan TF-IDF dan Support Vector Machine,” J. Inf. Technol., vol. 2, no. 1, pp. 36–40, Mar. 2022, doi: 10.46229/jifotech.v2i1.330.
[10] N. L. P. M. Putu, Ahmad Zuli Amrullah, and Ismarmiaty, “Analisis Sentimen dan Pemodelan Topik Pariwisata Lombok Menggunakan Algoritma Naive Bayes dan Latent Dirichlet Allocation,” J. RESTI (Rekayasa Sist. dan Teknol. Informasi), vol. 5, no. 1, pp. 123–131, 2021, doi: 10.29207/resti.v5i1.2587.
[11] M. I. Putri and I. Kharisudin, “Analisis Sentimen Pengguna Aplikasi Marketplace Tokopedia Pada Situs Google Play Menggunakan Metode Support Vector Machine (SVM), Naïve Bayes, dan Logistic Regression,” in PRISMA, Prosiding Seminar Nasional Matematika, 2022, vol. 5, pp. 759–766.
[12] Y. A. Sir and A. H. H. Soepranoto, “Pendekatan Resampling Data Untuk Menangani Masalah Ketidakseimbangan Kelas,” J. Komput. dan Inform., vol. 10, no. 1, pp. 31–38, Mar. 2022, doi: 10.35508/jicon.v10i1.6554.
[13] R. Khalida and S. Setiawati, “Analisis Sentimen Sistem E-Tilang Menggunakan Algoritma Naive Bayes Dengan Optimalisasi Information Gain,” J. Inform. Inf. Secur., vol. 1, no. 1, pp. 19–26, May 2020, doi: 10.31599/jiforty.v1i1.137.
[14] H. Hairani, A. Anggrawan, and D. Priyanto, “Improvement Performance of the Random Forest Method on Unbalanced Diabetes Data Classification Using Smote-Tomek Link,” Int. J. Informatics Vis., vol. 7, no. 1, pp. 258–264, 2023.
[15] H. Hairani and D. Priyanto, “A New Approach of Hybrid Sampling SMOTE and ENN to the Accuracy of Machine Learning Methods on Unbalanced Diabetes Disease Data,” Int. J. Adv. Comput. Sci. Appl., vol. 14, no. 8, pp. 585–590, 2023.
[16] E. F. Swana, W. Doorsamy, and P. Bokoro, “Tomek Link and SMOTE Approaches for Machine Fault Classification with an Imbalanced Dataset,” Sensors, vol. 22, no. 9, pp. 1–21, 2022, doi: 10.3390/s22093246.
[17] A. Fontanella, H. Ihsan, and Z. Wang, “Review of Accounting , Finance and Governance Sentiment Analysis of Indonesian Government Policy in the Environmental Sector Using Machine Learning Method,” J. Rafgo, vol. 1, no. 2, pp. 1–5, 2021.
[18] D. Pratmanto, R. Rousyati, F. F. Wati, A. E. Widodo, S. Suleman, and R. Wijianto, “App Review Sentiment Analysis Shopee Application in Google Play Store Using Naive Bayes Algorithm,” in Journal of Physics: Conference Series, 2020, vol. 1641, no. 1, pp. 1–7. doi: 10.1088/1742-6596/1641/1/012043.
[19] F. Y. A’la, “Indonesian Sentiment Analysis towards MyPertamina Application Reviews by Utilizing Machine Learning Algorithms,” J. Informatics, Inf. Syst. Softw. Eng. Appl., vol. 5, no. 1, pp. 80–91, 2022, doi: 10.20895/INISTA.V5I1.838.
[2] F. Fitriana, E. Utami, and H. Al Fatta, “Analisis Sentimen Opini Terhadap Vaksin Covid - 19 pada Media Sosial Twitter Menggunakan Support Vector Machine dan Naive Bayes,” J. Komtika (Komputasi dan Inform., vol. 5, no. 1, pp. 19–25, Jul. 2021, doi: 10.31603/komtika.v5i1.5185.
[3] P. T. Putra, A. Anggrawan, and H. Hairani, “Comparison of Machine Learning Methods for Classifying User Satisfaction Opinions of the PeduliLindungi Application,” Matrik J. Manajemen, Tek. Inform. dan Rekayasa Komput., vol. 22, no. 3, pp. 431–440, 2023, doi: 10.30812/matrik.v22i3.2860.
[4] H. Hairani, A. Anggrawan, A. I. Wathan, K. A. Latif, K. Marzuki, and M. Zulfikri, “The Abstract of Thesis Classifier by Using Naive Bayes Method,” in 2021 International Conference on Software Engineering & Computer Systems and 4th International Conference on Computational Science and Information Management (ICSECS-ICOCSIM), 2021, pp. 312–315. doi: 10.1109/ICSECS52883.2021.00063.
[5] J. W. Fernanda, “Pemodelan Persepsi Pembelajaran Online Menggunakan Latent Dirichlet Allocation,” J. Stat. Univ. Muhammadiyah Semarang, vol. 9, no. 2, pp. 79–85, 2021, doi: 10.26714/jsunimus.9.2.2021.79-85.
[6] M. F. Madjid, D. E. Ratnawati, and B. Rahayudi, “Sentiment Analysis on App Reviews Using Support Vector Machine and Naïve Bayes Classification,” Sink. J. dan Penelit. Tek. Inform., vol. 8, no. 1, pp. 556–562, 2023, [Online]. Available: https://jurnal.polgan.ac.id/index.php/sinkron/article/view/12161
[7] G. K. Locarso, “Analisis Sentimen Review Aplikasi Pedulilindungi Pada Google Play Store Menggunakan NBC,” J. Tek. Inform. Kaputama, vol. 6, no. 2, pp. 353–361, 2022.
[8] S. Rahayu, Y. MZ, J. E. Bororing, and R. Hadiyat, “Implementasi Metode K-Nearest Neighbor (K-NN) untuk Analisis Sentimen Kepuasan Pengguna Aplikasi Teknologi Finansial FLIP,” Edumatic J. Pendidik. Inform., vol. 6, no. 1, pp. 98–106, Jun. 2022, doi: 10.29408/edumatic.v6i1.5433.
[9] O. I. Gifari, M. Adha, F. Freddy, and F. F. S. Durrand, “Analisis Sentimen Review Film Menggunakan TF-IDF dan Support Vector Machine,” J. Inf. Technol., vol. 2, no. 1, pp. 36–40, Mar. 2022, doi: 10.46229/jifotech.v2i1.330.
[10] N. L. P. M. Putu, Ahmad Zuli Amrullah, and Ismarmiaty, “Analisis Sentimen dan Pemodelan Topik Pariwisata Lombok Menggunakan Algoritma Naive Bayes dan Latent Dirichlet Allocation,” J. RESTI (Rekayasa Sist. dan Teknol. Informasi), vol. 5, no. 1, pp. 123–131, 2021, doi: 10.29207/resti.v5i1.2587.
[11] M. I. Putri and I. Kharisudin, “Analisis Sentimen Pengguna Aplikasi Marketplace Tokopedia Pada Situs Google Play Menggunakan Metode Support Vector Machine (SVM), Naïve Bayes, dan Logistic Regression,” in PRISMA, Prosiding Seminar Nasional Matematika, 2022, vol. 5, pp. 759–766.
[12] Y. A. Sir and A. H. H. Soepranoto, “Pendekatan Resampling Data Untuk Menangani Masalah Ketidakseimbangan Kelas,” J. Komput. dan Inform., vol. 10, no. 1, pp. 31–38, Mar. 2022, doi: 10.35508/jicon.v10i1.6554.
[13] R. Khalida and S. Setiawati, “Analisis Sentimen Sistem E-Tilang Menggunakan Algoritma Naive Bayes Dengan Optimalisasi Information Gain,” J. Inform. Inf. Secur., vol. 1, no. 1, pp. 19–26, May 2020, doi: 10.31599/jiforty.v1i1.137.
[14] H. Hairani, A. Anggrawan, and D. Priyanto, “Improvement Performance of the Random Forest Method on Unbalanced Diabetes Data Classification Using Smote-Tomek Link,” Int. J. Informatics Vis., vol. 7, no. 1, pp. 258–264, 2023.
[15] H. Hairani and D. Priyanto, “A New Approach of Hybrid Sampling SMOTE and ENN to the Accuracy of Machine Learning Methods on Unbalanced Diabetes Disease Data,” Int. J. Adv. Comput. Sci. Appl., vol. 14, no. 8, pp. 585–590, 2023.
[16] E. F. Swana, W. Doorsamy, and P. Bokoro, “Tomek Link and SMOTE Approaches for Machine Fault Classification with an Imbalanced Dataset,” Sensors, vol. 22, no. 9, pp. 1–21, 2022, doi: 10.3390/s22093246.
[17] A. Fontanella, H. Ihsan, and Z. Wang, “Review of Accounting , Finance and Governance Sentiment Analysis of Indonesian Government Policy in the Environmental Sector Using Machine Learning Method,” J. Rafgo, vol. 1, no. 2, pp. 1–5, 2021.
[18] D. Pratmanto, R. Rousyati, F. F. Wati, A. E. Widodo, S. Suleman, and R. Wijianto, “App Review Sentiment Analysis Shopee Application in Google Play Store Using Naive Bayes Algorithm,” in Journal of Physics: Conference Series, 2020, vol. 1641, no. 1, pp. 1–7. doi: 10.1088/1742-6596/1641/1/012043.
[19] F. Y. A’la, “Indonesian Sentiment Analysis towards MyPertamina Application Reviews by Utilizing Machine Learning Algorithms,” J. Informatics, Inf. Syst. Softw. Eng. Appl., vol. 5, no. 1, pp. 80–91, 2022, doi: 10.20895/INISTA.V5I1.838.
Published
2023-09-28
How to Cite
[1]
I. N. Switrayana, D. Ashadi, H. Hairani, and A. Aminuddin, “Sentiment Analysis and Topic Modeling of Kitabisa Applications using Support Vector Machine (SVM) and Smote-Tomek Links Methods”, International Journal of Engineering and Computer Science Applications (IJECSA), vol. 2, no. 2, pp. 81 - 91, Sep. 2023.
Section
Articles
