Sentiment Analysis and Topic Modeling of Kitabisa Applications using Support Vector Machine (SVM) and Smote-Tomek Links Methods
DOI:
https://doi.org/10.30812/ijecsa.v2i2.3406Keywords:
Sentiment Analysis, Support Vector Machine, SMOTE-Tomek Links, Latent Dirichlet AllocationAbstract
Kitabisa is an Indonesian application that functions to raise funds online. Users can easily support various types of campaigns and donate funds to various social causes through the app. User reviews of the application are very diverse, and it is not sure whether user reviews of the application tend to be positive, neutral, or negative. This research aimed to analyze the sentiment of the Kitabisa application by modeling topics using Latent Dirichlet Allocation (LDA) and classifying user reviews using a Support Vector Machine (SVM). The scrapped dataset showed imbalanced dataset problems, so the SMOTE-Tomek Links oversampling technique was proposed. The results of this study show that using LDA produces five topics often discussed in 750 reviews. Then, the performance of SVM without using SMOTE-Tomek Links was 72% accuracy, 76% precision, 72% recall, and 64% f1 score. Meanwhile, using SMOTE-Tomek Links could significantly improve the performance, namely 98% accuracy, 98% precision, 98% recall, and 98% f1 score. Based on this research, the application of SVM achieved high performance for user sentiment classification, especially when the dataset was in a balanced state. Therefore, the SMOTE-Tomek Links oversampling technique is recommended for dealing with unbalanced sentiment datasets.
References
I. Irwansyah and N. Hutami, “Pemanfaatan Aplikasi Mobile Kitabisa Dalam Pelaksanaan Crowdfunding Di Indonesia.,†J. Komun., vol. 13, no. 2, pp. 183–194, 2019, doi: https://doi.org/10.21107/ilkom.v13i2.5357.
F. Fitriana, E. Utami, and H. Al Fatta, “Analisis Sentimen Opini Terhadap Vaksin Covid - 19 pada Media Sosial Twitter Menggunakan Support Vector Machine dan Naive Bayes,†J. Komtika (Komputasi dan Inform., vol. 5, no. 1, pp. 19–25, Jul. 2021, doi: 10.31603/komtika.v5i1.5185.
P. T. Putra, A. Anggrawan, and H. Hairani, “Comparison of Machine Learning Methods for Classifying User Satisfaction Opinions of the PeduliLindungi Application,†Matrik J. Manajemen, Tek. Inform. dan Rekayasa Komput., vol. 22, no. 3, pp. 431–440, 2023, doi: 10.30812/matrik.v22i3.2860.
H. Hairani, A. Anggrawan, A. I. Wathan, K. A. Latif, K. Marzuki, and M. Zulfikri, “The Abstract of Thesis Classifier by Using Naive Bayes Method,†in 2021 International Conference on Software Engineering & Computer Systems and 4th International Conference on Computational Science and Information Management (ICSECS-ICOCSIM), 2021, pp. 312–315. doi: 10.1109/ICSECS52883.2021.00063.
J. W. Fernanda, “Pemodelan Persepsi Pembelajaran Online Menggunakan Latent Dirichlet Allocation,†J. Stat. Univ. Muhammadiyah Semarang, vol. 9, no. 2, pp. 79–85, 2021, doi: 10.26714/jsunimus.9.2.2021.79-85.
M. F. Madjid, D. E. Ratnawati, and B. Rahayudi, “Sentiment Analysis on App Reviews Using Support Vector Machine and Naïve Bayes Classification,†Sink. J. dan Penelit. Tek. Inform., vol. 8, no. 1, pp. 556–562, 2023, [Online]. Available: https://jurnal.polgan.ac.id/index.php/sinkron/article/view/12161
G. K. Locarso, “Analisis Sentimen Review Aplikasi Pedulilindungi Pada Google Play Store Menggunakan NBC,†J. Tek. Inform. Kaputama, vol. 6, no. 2, pp. 353–361, 2022.
S. Rahayu, Y. MZ, J. E. Bororing, and R. Hadiyat, “Implementasi Metode K-Nearest Neighbor (K-NN) untuk Analisis Sentimen Kepuasan Pengguna Aplikasi Teknologi Finansial FLIP,†Edumatic J. Pendidik. Inform., vol. 6, no. 1, pp. 98–106, Jun. 2022, doi: 10.29408/edumatic.v6i1.5433.
O. I. Gifari, M. Adha, F. Freddy, and F. F. S. Durrand, “Analisis Sentimen Review Film Menggunakan TF-IDF dan Support Vector Machine,†J. Inf. Technol., vol. 2, no. 1, pp. 36–40, Mar. 2022, doi: 10.46229/jifotech.v2i1.330.
N. L. P. M. Putu, Ahmad Zuli Amrullah, and Ismarmiaty, “Analisis Sentimen dan Pemodelan Topik Pariwisata Lombok Menggunakan Algoritma Naive Bayes dan Latent Dirichlet Allocation,†J. RESTI (Rekayasa Sist. dan Teknol. Informasi), vol. 5, no. 1, pp. 123–131, 2021, doi: 10.29207/resti.v5i1.2587.
M. I. Putri and I. Kharisudin, “Analisis Sentimen Pengguna Aplikasi Marketplace Tokopedia Pada Situs Google Play Menggunakan Metode Support Vector Machine (SVM), Naïve Bayes, dan Logistic Regression,†in PRISMA, Prosiding Seminar Nasional Matematika, 2022, vol. 5, pp. 759–766.
Y. A. Sir and A. H. H. Soepranoto, “Pendekatan Resampling Data Untuk Menangani Masalah Ketidakseimbangan Kelas,†J. Komput. dan Inform., vol. 10, no. 1, pp. 31–38, Mar. 2022, doi: 10.35508/jicon.v10i1.6554.
R. Khalida and S. Setiawati, “Analisis Sentimen Sistem E-Tilang Menggunakan Algoritma Naive Bayes Dengan Optimalisasi Information Gain,†J. Inform. Inf. Secur., vol. 1, no. 1, pp. 19–26, May 2020, doi: 10.31599/jiforty.v1i1.137.
H. Hairani, A. Anggrawan, and D. Priyanto, “Improvement Performance of the Random Forest Method on Unbalanced Diabetes Data Classification Using Smote-Tomek Link,†Int. J. Informatics Vis., vol. 7, no. 1, pp. 258–264, 2023.
H. Hairani and D. Priyanto, “A New Approach of Hybrid Sampling SMOTE and ENN to the Accuracy of Machine Learning Methods on Unbalanced Diabetes Disease Data,†Int. J. Adv. Comput. Sci. Appl., vol. 14, no. 8, pp. 585–590, 2023.
E. F. Swana, W. Doorsamy, and P. Bokoro, “Tomek Link and SMOTE Approaches for Machine Fault Classification with an Imbalanced Dataset,†Sensors, vol. 22, no. 9, pp. 1–21, 2022, doi: 10.3390/s22093246.
A. Fontanella, H. Ihsan, and Z. Wang, “Review of Accounting , Finance and Governance Sentiment Analysis of Indonesian Government Policy in the Environmental Sector Using Machine Learning Method,†J. Rafgo, vol. 1, no. 2, pp. 1–5, 2021.
D. Pratmanto, R. Rousyati, F. F. Wati, A. E. Widodo, S. Suleman, and R. Wijianto, “App Review Sentiment Analysis Shopee Application in Google Play Store Using Naive Bayes Algorithm,†in Journal of Physics: Conference Series, 2020, vol. 1641, no. 1, pp. 1–7. doi: 10.1088/1742-6596/1641/1/012043.
F. Y. A’la, “Indonesian Sentiment Analysis towards MyPertamina Application Reviews by Utilizing Machine Learning Algorithms,†J. Informatics, Inf. Syst. Softw. Eng. Appl., vol. 5, no. 1, pp. 80–91, 2022, doi: 10.20895/INISTA.V5I1.838.
F. Fitriana, E. Utami, and H. Al Fatta, “Analisis Sentimen Opini Terhadap Vaksin Covid - 19 pada Media Sosial Twitter Menggunakan Support Vector Machine dan Naive Bayes,†J. Komtika (Komputasi dan Inform., vol. 5, no. 1, pp. 19–25, Jul. 2021, doi: 10.31603/komtika.v5i1.5185.
P. T. Putra, A. Anggrawan, and H. Hairani, “Comparison of Machine Learning Methods for Classifying User Satisfaction Opinions of the PeduliLindungi Application,†Matrik J. Manajemen, Tek. Inform. dan Rekayasa Komput., vol. 22, no. 3, pp. 431–440, 2023, doi: 10.30812/matrik.v22i3.2860.
H. Hairani, A. Anggrawan, A. I. Wathan, K. A. Latif, K. Marzuki, and M. Zulfikri, “The Abstract of Thesis Classifier by Using Naive Bayes Method,†in 2021 International Conference on Software Engineering & Computer Systems and 4th International Conference on Computational Science and Information Management (ICSECS-ICOCSIM), 2021, pp. 312–315. doi: 10.1109/ICSECS52883.2021.00063.
J. W. Fernanda, “Pemodelan Persepsi Pembelajaran Online Menggunakan Latent Dirichlet Allocation,†J. Stat. Univ. Muhammadiyah Semarang, vol. 9, no. 2, pp. 79–85, 2021, doi: 10.26714/jsunimus.9.2.2021.79-85.
M. F. Madjid, D. E. Ratnawati, and B. Rahayudi, “Sentiment Analysis on App Reviews Using Support Vector Machine and Naïve Bayes Classification,†Sink. J. dan Penelit. Tek. Inform., vol. 8, no. 1, pp. 556–562, 2023, [Online]. Available: https://jurnal.polgan.ac.id/index.php/sinkron/article/view/12161
G. K. Locarso, “Analisis Sentimen Review Aplikasi Pedulilindungi Pada Google Play Store Menggunakan NBC,†J. Tek. Inform. Kaputama, vol. 6, no. 2, pp. 353–361, 2022.
S. Rahayu, Y. MZ, J. E. Bororing, and R. Hadiyat, “Implementasi Metode K-Nearest Neighbor (K-NN) untuk Analisis Sentimen Kepuasan Pengguna Aplikasi Teknologi Finansial FLIP,†Edumatic J. Pendidik. Inform., vol. 6, no. 1, pp. 98–106, Jun. 2022, doi: 10.29408/edumatic.v6i1.5433.
O. I. Gifari, M. Adha, F. Freddy, and F. F. S. Durrand, “Analisis Sentimen Review Film Menggunakan TF-IDF dan Support Vector Machine,†J. Inf. Technol., vol. 2, no. 1, pp. 36–40, Mar. 2022, doi: 10.46229/jifotech.v2i1.330.
N. L. P. M. Putu, Ahmad Zuli Amrullah, and Ismarmiaty, “Analisis Sentimen dan Pemodelan Topik Pariwisata Lombok Menggunakan Algoritma Naive Bayes dan Latent Dirichlet Allocation,†J. RESTI (Rekayasa Sist. dan Teknol. Informasi), vol. 5, no. 1, pp. 123–131, 2021, doi: 10.29207/resti.v5i1.2587.
M. I. Putri and I. Kharisudin, “Analisis Sentimen Pengguna Aplikasi Marketplace Tokopedia Pada Situs Google Play Menggunakan Metode Support Vector Machine (SVM), Naïve Bayes, dan Logistic Regression,†in PRISMA, Prosiding Seminar Nasional Matematika, 2022, vol. 5, pp. 759–766.
Y. A. Sir and A. H. H. Soepranoto, “Pendekatan Resampling Data Untuk Menangani Masalah Ketidakseimbangan Kelas,†J. Komput. dan Inform., vol. 10, no. 1, pp. 31–38, Mar. 2022, doi: 10.35508/jicon.v10i1.6554.
R. Khalida and S. Setiawati, “Analisis Sentimen Sistem E-Tilang Menggunakan Algoritma Naive Bayes Dengan Optimalisasi Information Gain,†J. Inform. Inf. Secur., vol. 1, no. 1, pp. 19–26, May 2020, doi: 10.31599/jiforty.v1i1.137.
H. Hairani, A. Anggrawan, and D. Priyanto, “Improvement Performance of the Random Forest Method on Unbalanced Diabetes Data Classification Using Smote-Tomek Link,†Int. J. Informatics Vis., vol. 7, no. 1, pp. 258–264, 2023.
H. Hairani and D. Priyanto, “A New Approach of Hybrid Sampling SMOTE and ENN to the Accuracy of Machine Learning Methods on Unbalanced Diabetes Disease Data,†Int. J. Adv. Comput. Sci. Appl., vol. 14, no. 8, pp. 585–590, 2023.
E. F. Swana, W. Doorsamy, and P. Bokoro, “Tomek Link and SMOTE Approaches for Machine Fault Classification with an Imbalanced Dataset,†Sensors, vol. 22, no. 9, pp. 1–21, 2022, doi: 10.3390/s22093246.
A. Fontanella, H. Ihsan, and Z. Wang, “Review of Accounting , Finance and Governance Sentiment Analysis of Indonesian Government Policy in the Environmental Sector Using Machine Learning Method,†J. Rafgo, vol. 1, no. 2, pp. 1–5, 2021.
D. Pratmanto, R. Rousyati, F. F. Wati, A. E. Widodo, S. Suleman, and R. Wijianto, “App Review Sentiment Analysis Shopee Application in Google Play Store Using Naive Bayes Algorithm,†in Journal of Physics: Conference Series, 2020, vol. 1641, no. 1, pp. 1–7. doi: 10.1088/1742-6596/1641/1/012043.
F. Y. A’la, “Indonesian Sentiment Analysis towards MyPertamina Application Reviews by Utilizing Machine Learning Algorithms,†J. Informatics, Inf. Syst. Softw. Eng. Appl., vol. 5, no. 1, pp. 80–91, 2022, doi: 10.20895/INISTA.V5I1.838.
Downloads
Published
2023-09-28
Issue
Section
Articles
How to Cite
[1]
I. N. Switrayana, D. Ashadi, H. Hairani, and A. Aminuddin, “Sentiment Analysis and Topic Modeling of Kitabisa Applications using Support Vector Machine (SVM) and Smote-Tomek Links Methods”, IJECSA, vol. 2, no. 2, pp. 87–98, Sep. 2023, doi: 10.30812/ijecsa.v2i2.3406.
Most read articles by the same author(s)
- Christopher Michael Lauw, Hairani Hairani, Ilham Saifuddin, Juvinal Ximenes Guterres, Muhammad Maariful Huda, Mayadi Mayadi, Combination of Smote and Random Forest Methods for Lung Cancer Classification , International Journal of Engineering and Computer Science Applications (IJECSA): Vol. 2 No. 2 (2023): September 2023
- Hairani Hairani, Mengas Janhasmadja, Abu Tholib, Juvinal Ximenes Guterres, Yuri Ariyanto, Thesis Topic Modeling Study: Latent Dirichlet Allocation (LDA) and Machine Learning Approach , International Journal of Engineering and Computer Science Applications (IJECSA): Vol. 3 No. 2 (2024): September 2024
- Riosatria Riosatria, Hairani Hairani, Anthony Anggrawan, Moch. Syahrir, Enhancing Mental Illness Predictions: Analyzing Trends Using Multiple Linear Regression and Neural Network Backpropagation , International Journal of Engineering and Computer Science Applications (IJECSA): Vol. 3 No. 2 (2024): September 2024
- Hairani Hairani, Juvinal Ximenes Guterres, Exploring Customer Purchasing Patterns: A Study Utilizing FP-Growth Algorithm on Supermarket Transaction Data , International Journal of Engineering and Computer Science Applications (IJECSA): Vol. 3 No. 1 (2024): March 2024
- Hairani Hairani, Lilik Nurhayati, Muhammad Innuddin, Web-Based Application for Toddler Nutrition Classification Using C4.5 Algorithm , International Journal of Engineering and Computer Science Applications (IJECSA): Vol. 1 No. 2 (2022): September 2022
- Dias Nabila Huda, Anthony Anggrawan, Hairani Hairani, Clustering Analysis of Umrah Pilgrim Data Based on the K-Medoid Method , International Journal of Engineering and Computer Science Applications (IJECSA): Vol. 3 No. 2 (2024): September 2024
- Anthony Anggrawan, Hairani Hairani, M. Ade Candra, Prediction of Electricity Usage with Back-propagation Neural Network , International Journal of Engineering and Computer Science Applications (IJECSA): Vol. 1 No. 1 (2022): March 2022
- Ramadhanti Ramadhanti, Hairani Hairani, Muhammad Innuddin, Electric Vehicle Sales-Prediction Application Using Backpropagation Algorithm Based on Web , International Journal of Engineering and Computer Science Applications (IJECSA): Vol. 2 No. 2 (2023): September 2023
