Comparison of Machine Learning Methods for Classifying User Satisfaction Opinions of the PeduliLindungi Application
Keywords:
Machine Learning, PeduliLindungi Application, Sentiment Analysis, Text Mining
Abstract
Since the emergence of the Covid-19 virus, the Indonesian government urged people to study, work, and worship or work from home. The social restriction policy has changed people's behavior which requires physical distance in social interaction. The government developed an application to minimize the spread of Covid-19, namely the PeduliLindungi application. The PeduliLindungi application is a tracking application to prevent the spread of Covid-19. The government's policy of implementing the PeduliLindungi application during Covid-19 aroused pros and cons from the public. The volume of PeduliLindungi application review data on Google Play was increasing, so manual analysis could not be done. New analytical approaches needed to be carried out, such as sentiment analysis. This research aimed to analyze user reviews of the PeduilLindungi application using classification methods, namely Support Vector Machine (SVM), Random Forest, and Naïve Bayes. The methods used were Synthetic Minority Oversampling Technique (SMOTE), Random Forest, SVM, and Naïve Bayes. SMOTE was used to balance user review data on the PeduliLindungi application. After the data had been balanced, classification was carried out. The results of this study showed that the Random Forest method with SMOTE got better accuracy than the SVM and Naive Bayes methods, which was 96.3% based on the division of training and testing data using 10-fold cross-validation. Thus, using the SMOTE method could improve the accuracy of classification methods in classifying opinions of user satisfaction with the PeduliLindungi application.
Downloads
Download data is not yet available.
References
[1] L. Ellyanti, Y. Ruldeviyani, L. E. Pradana, and A. Harjanto, "Sentiment Analysis of Twitter Users to the PeduliLindungi Using Naïve Bayes Algorithm," J. RESTI (Rekayasa Sist. dan Teknol. Informasi), vol. 7, no. 2, pp. 414–421, 2023.
[2] M. I. Nurmansyah, C. Rosidati, Yustiyani, and N. M. Nasir, “Measuring the Success of PeduliLindungi Application Use for Supporting COVID-19 Prevention: A Case Study among College Students in Jakarta, Indonesia,” Kesmas J. Kesehat. Masy. Nas., vol. 17, no. 1, pp. 11–16, 2022, doi: 10.21109/KESMAS.V17ISP1.6057.
[3] W. Sudiarsa and G. B. Wiraditya, "Heuristic Evaluation Usability Analisys on Information and Tracking Covid-19 Application Peduli Lindungi Using Heuristic Evaluation," J. Inf. Technol. Comput. Sci., vol. 3, no. 2, pp. 354–364, 2020.
[4] R. D. Novita and M. A. Razak, "Personal Data Protection in Falsification of Covid-19 Vaccination: A Juridical Review," YURIS (Journal Court Justice), vol. 1, no. 3, pp. 25–37, 2022.
[5] Z. Drus and H. Khalid, "Sentiment Analysis in Social Media and Its Application: Systematic Literature Review," Procedia Comput. Sci., vol. 161, pp. 707–714, 2019, doi: 10.1016/j.procs.2019.11.174.
[6] G. K. Locarso, “Analisis Sentimen Review Aplikasi Pedulilindungi Pada Google Play Store Menggunakan NBC,” J. Tek. Inform. Kaputama, vol. 6, no. 2, pp. 353–361, 2022.
[7] I. Yunanto and S. Yulianto, “Twitter Sentiment Analysis Pedulilindungi Application Using Naïve Bayes and Support Vector Machine,” J. Tek. Inform., vol. 3, no. 4, pp. 807–814, 2022, doi: 10.20884/1.jutif.2022.3.4.292.
[8] A. Salma and W. Silfianti, "Sentiment Analysis of User Review on COVID-19 Information Applications Using Naïve Bayes Classifier, Support Vector Machine, and K-Nearest Neighbors," Int. Res. J. Adv. Eng. Sci., vol. 6, no. 4, pp. 158–162, 2021.
[9] Z. Rais, F. T. T. Hakiki, and R. Aprianti, "Sentiment Analysis of Peduli Lindungi Application Using the Naive Bayes Method," SAINSMAT J. Appl. Sci. Math. Its Educ., vol. 11, no. 1, pp. 23–29, 2022, doi: 10.35877/sainsmat794.
[10] G. A. Lustiansyah et al., “Analisis klasifikasi sentimen pengguna aplikasi pedulilindungi berdasarkan ulasan dengan menggunakan metode long short term memory,” in Seminar Nasional Mahasiswa Ilmu Komputer dan Aplikasinya (SENAMIKA), 2022, pp. 327–336.
[11] I. Saputra, T. Djatna, R. R. A. Siregar, D. A. Kristiyanti, H. R. Yani, and A. A. Riyadi, “Text Mining of PeduliLindungi Application Reviews on Google Play Store,” Fakt. Exacta, vol. 15, no. 2, pp. 101–108, 2022.
[12] A. Mustopa, Hermanto, Anna, E. B. Pratama, A. Hendini, and D. Risdiansyah, "Analysis of user reviews for the pedulilindungi application on google play using the support vector machine and naive bayes algorithm based on particle swarm optimization," in 2020 5th International Conference on Informatics and Computing, ICIC 2020, 2020, vol. 2, pp. 1–7. doi: 10.1109/ICIC50835.2020.9288655.
[13] P. A. Aritonang, M. E. Johan, and I. Prasetiawan, “Aspect-Based Sentiment Analysis on Application Review using CNN (Case Study : Peduli Lindungi Application),” Ultim. Infosys J. Ilmu Sist. Inf., vol. 13, no. 1, pp. 54–61, 2022.
[14] F. S. Darusman, A. A. Arifiyanti, S. Fitri, and A. Wati, “Sentiment Analysis Pedulilindungi Tweet Using Support Vector Machine Method,” Appl. Technol. Comput. Sci. J., vol. 4, no. 2, pp. 113–118, 2022.
[15] H. Hairani, L. Nurhayati, and M. Innuddin, "Web-Based Application for Toddler Nutrition Classification Using C4.5 Algorithm," Int. J. Eng. Comput. Sci. Appl., vol. 1, no. 2, pp. 77–82, 2022, doi: 10.30812/ijecsa.v1i2.2387.
[16] U. Kusnia and F. Kurniawan, “Analisis Sentimen Review Aplikasi Media Berita Online Pada Google Play menggunakan Metode Algoritma Support Vector Machines ( SVM ) Dan Naive Bayes,” J. Keilmuan Apl. Tek. Inform., vol. 5, no. 36, pp. 24–28, 2022, doi: https://doi.org/10.35891/explorit.v14i1.3116.
[17] F. Romadoni, Y. Umaidah, and B. N. Sari, “Text Mining Untuk Analisis Sentimen Pelanggan Terhadap Layanan Uang Elektronik Menggunakan Algoritma Support Vector Machine,” J. Sisfokom (Sistem Inf. dan Komputer), vol. 9, no. 2, pp. 247–253, 2020, doi: 10.32736/sisfokom.v9i2.903.
[18] T. C. Herdiyani and A. U. Zailani, “Sentiment Analysis Terkait Pemindahan Ibu Kota Indonesia Menggunakan Metode Random Forest Berdasarkan Tweet Warga Negara Indonesia,” J. Teknol. Sist. Inf., vol. 3, no. 2, pp. 154–165, Sep. 2022, doi: 10.35957/jtsi.v3i2.2920.
[19] H. Hairani, A. Anggrawan, A. I. Wathan, K. A. Latif, K. Marzuki, and M. Zulfikri, "The Abstract of Thesis Classifier by Using Naive Bayes Method," in 2021 International Conference on Software Engineering & Computer Systems and 4th International Conference on Computational Science and Information Management (ICSECS-ICOCSIM), 2021, pp. 312–315. doi: 10.1109/ICSECS52883.2021.00063.
[20] H. P. Doloksaribu and Y. T. Samuel, “Komparasi Algoritma Data Mining Untuk Analisis Sentimen Aplikasi Pedulilindungi,” J. Teknol. Inf. J. Keilmuan dan Apl. Bid. Tek. Inform., vol. 16, no. 1, pp. 1–11, 2022.
[21] N. Muna et al., “Penerapan Algoritma Random Forest untuk Analisis Sentimen Komentar di YouTube tentang Islamfobia,” J. Nas. Komputasi dan Teknol. Inf., vol. 5, no. 1, pp. 49–54, 2019.
[22] V. Rupapara, F. Rustam, H. F. Shahzad, A. Mehmood, I. Ashraf, and G. S. Choi, "Impact of SMOTE on Imbalanced Text Features for Toxic Comments Classification Using RVVC Model," IEEE Access, vol. 9, pp. 78621–78634, 2021, doi: 10.1109/ACCESS.2021.3083638.
[23] T. Wongvorachan, S. He, and O. Bulut, "A Comparison of Undersampling , Oversampling , and SMOTE Methods for Dealing with Imbalanced Classification in Educational Data Mining," Information, vol. 14, no. 1, pp. 1–15, 2023.
[24] H. Hairani, A. S. Suweleh, and D. Susilowaty, “Penanganan Ketidak Seimbangan Kelas Menggunakan Pendekatan Level Data,” MATRIK J. Manajemen, Tek. Inform. dan Rekayasa Komput., vol. 20, no. 1, pp. 109–116, 2020, doi: 10.30812/matrik.v20i1.846.
[25] H. Hairani, A. Anggrawan, and D. Priyanto, "Improvement Performance of the Random Forest Method on Unbalanced Diabetes Data Classification Using Smote-Tomek Link," Int. J. Informatics Vis., vol. 7, no. 1, pp. 258–264, 2023.
[26] A. Anggrawan, H. Hairani, and C. Satria, "Improving SVM Classification Performance on Unbalanced Student Graduation Time Data Using SMOTE," Int. J. Inf. Educ. Technol., vol. 13, no. 2, pp. 289–295, 2023, doi: 10.18178/ijiet.2023.13.2.1806.
[2] M. I. Nurmansyah, C. Rosidati, Yustiyani, and N. M. Nasir, “Measuring the Success of PeduliLindungi Application Use for Supporting COVID-19 Prevention: A Case Study among College Students in Jakarta, Indonesia,” Kesmas J. Kesehat. Masy. Nas., vol. 17, no. 1, pp. 11–16, 2022, doi: 10.21109/KESMAS.V17ISP1.6057.
[3] W. Sudiarsa and G. B. Wiraditya, "Heuristic Evaluation Usability Analisys on Information and Tracking Covid-19 Application Peduli Lindungi Using Heuristic Evaluation," J. Inf. Technol. Comput. Sci., vol. 3, no. 2, pp. 354–364, 2020.
[4] R. D. Novita and M. A. Razak, "Personal Data Protection in Falsification of Covid-19 Vaccination: A Juridical Review," YURIS (Journal Court Justice), vol. 1, no. 3, pp. 25–37, 2022.
[5] Z. Drus and H. Khalid, "Sentiment Analysis in Social Media and Its Application: Systematic Literature Review," Procedia Comput. Sci., vol. 161, pp. 707–714, 2019, doi: 10.1016/j.procs.2019.11.174.
[6] G. K. Locarso, “Analisis Sentimen Review Aplikasi Pedulilindungi Pada Google Play Store Menggunakan NBC,” J. Tek. Inform. Kaputama, vol. 6, no. 2, pp. 353–361, 2022.
[7] I. Yunanto and S. Yulianto, “Twitter Sentiment Analysis Pedulilindungi Application Using Naïve Bayes and Support Vector Machine,” J. Tek. Inform., vol. 3, no. 4, pp. 807–814, 2022, doi: 10.20884/1.jutif.2022.3.4.292.
[8] A. Salma and W. Silfianti, "Sentiment Analysis of User Review on COVID-19 Information Applications Using Naïve Bayes Classifier, Support Vector Machine, and K-Nearest Neighbors," Int. Res. J. Adv. Eng. Sci., vol. 6, no. 4, pp. 158–162, 2021.
[9] Z. Rais, F. T. T. Hakiki, and R. Aprianti, "Sentiment Analysis of Peduli Lindungi Application Using the Naive Bayes Method," SAINSMAT J. Appl. Sci. Math. Its Educ., vol. 11, no. 1, pp. 23–29, 2022, doi: 10.35877/sainsmat794.
[10] G. A. Lustiansyah et al., “Analisis klasifikasi sentimen pengguna aplikasi pedulilindungi berdasarkan ulasan dengan menggunakan metode long short term memory,” in Seminar Nasional Mahasiswa Ilmu Komputer dan Aplikasinya (SENAMIKA), 2022, pp. 327–336.
[11] I. Saputra, T. Djatna, R. R. A. Siregar, D. A. Kristiyanti, H. R. Yani, and A. A. Riyadi, “Text Mining of PeduliLindungi Application Reviews on Google Play Store,” Fakt. Exacta, vol. 15, no. 2, pp. 101–108, 2022.
[12] A. Mustopa, Hermanto, Anna, E. B. Pratama, A. Hendini, and D. Risdiansyah, "Analysis of user reviews for the pedulilindungi application on google play using the support vector machine and naive bayes algorithm based on particle swarm optimization," in 2020 5th International Conference on Informatics and Computing, ICIC 2020, 2020, vol. 2, pp. 1–7. doi: 10.1109/ICIC50835.2020.9288655.
[13] P. A. Aritonang, M. E. Johan, and I. Prasetiawan, “Aspect-Based Sentiment Analysis on Application Review using CNN (Case Study : Peduli Lindungi Application),” Ultim. Infosys J. Ilmu Sist. Inf., vol. 13, no. 1, pp. 54–61, 2022.
[14] F. S. Darusman, A. A. Arifiyanti, S. Fitri, and A. Wati, “Sentiment Analysis Pedulilindungi Tweet Using Support Vector Machine Method,” Appl. Technol. Comput. Sci. J., vol. 4, no. 2, pp. 113–118, 2022.
[15] H. Hairani, L. Nurhayati, and M. Innuddin, "Web-Based Application for Toddler Nutrition Classification Using C4.5 Algorithm," Int. J. Eng. Comput. Sci. Appl., vol. 1, no. 2, pp. 77–82, 2022, doi: 10.30812/ijecsa.v1i2.2387.
[16] U. Kusnia and F. Kurniawan, “Analisis Sentimen Review Aplikasi Media Berita Online Pada Google Play menggunakan Metode Algoritma Support Vector Machines ( SVM ) Dan Naive Bayes,” J. Keilmuan Apl. Tek. Inform., vol. 5, no. 36, pp. 24–28, 2022, doi: https://doi.org/10.35891/explorit.v14i1.3116.
[17] F. Romadoni, Y. Umaidah, and B. N. Sari, “Text Mining Untuk Analisis Sentimen Pelanggan Terhadap Layanan Uang Elektronik Menggunakan Algoritma Support Vector Machine,” J. Sisfokom (Sistem Inf. dan Komputer), vol. 9, no. 2, pp. 247–253, 2020, doi: 10.32736/sisfokom.v9i2.903.
[18] T. C. Herdiyani and A. U. Zailani, “Sentiment Analysis Terkait Pemindahan Ibu Kota Indonesia Menggunakan Metode Random Forest Berdasarkan Tweet Warga Negara Indonesia,” J. Teknol. Sist. Inf., vol. 3, no. 2, pp. 154–165, Sep. 2022, doi: 10.35957/jtsi.v3i2.2920.
[19] H. Hairani, A. Anggrawan, A. I. Wathan, K. A. Latif, K. Marzuki, and M. Zulfikri, "The Abstract of Thesis Classifier by Using Naive Bayes Method," in 2021 International Conference on Software Engineering & Computer Systems and 4th International Conference on Computational Science and Information Management (ICSECS-ICOCSIM), 2021, pp. 312–315. doi: 10.1109/ICSECS52883.2021.00063.
[20] H. P. Doloksaribu and Y. T. Samuel, “Komparasi Algoritma Data Mining Untuk Analisis Sentimen Aplikasi Pedulilindungi,” J. Teknol. Inf. J. Keilmuan dan Apl. Bid. Tek. Inform., vol. 16, no. 1, pp. 1–11, 2022.
[21] N. Muna et al., “Penerapan Algoritma Random Forest untuk Analisis Sentimen Komentar di YouTube tentang Islamfobia,” J. Nas. Komputasi dan Teknol. Inf., vol. 5, no. 1, pp. 49–54, 2019.
[22] V. Rupapara, F. Rustam, H. F. Shahzad, A. Mehmood, I. Ashraf, and G. S. Choi, "Impact of SMOTE on Imbalanced Text Features for Toxic Comments Classification Using RVVC Model," IEEE Access, vol. 9, pp. 78621–78634, 2021, doi: 10.1109/ACCESS.2021.3083638.
[23] T. Wongvorachan, S. He, and O. Bulut, "A Comparison of Undersampling , Oversampling , and SMOTE Methods for Dealing with Imbalanced Classification in Educational Data Mining," Information, vol. 14, no. 1, pp. 1–15, 2023.
[24] H. Hairani, A. S. Suweleh, and D. Susilowaty, “Penanganan Ketidak Seimbangan Kelas Menggunakan Pendekatan Level Data,” MATRIK J. Manajemen, Tek. Inform. dan Rekayasa Komput., vol. 20, no. 1, pp. 109–116, 2020, doi: 10.30812/matrik.v20i1.846.
[25] H. Hairani, A. Anggrawan, and D. Priyanto, "Improvement Performance of the Random Forest Method on Unbalanced Diabetes Data Classification Using Smote-Tomek Link," Int. J. Informatics Vis., vol. 7, no. 1, pp. 258–264, 2023.
[26] A. Anggrawan, H. Hairani, and C. Satria, "Improving SVM Classification Performance on Unbalanced Student Graduation Time Data Using SMOTE," Int. J. Inf. Educ. Technol., vol. 13, no. 2, pp. 289–295, 2023, doi: 10.18178/ijiet.2023.13.2.1806.
Published
2023-06-16
How to Cite
Tisna Putra, P., Anggrawan, A., & Hairani, H. (2023). Comparison of Machine Learning Methods for Classifying User Satisfaction Opinions of the PeduliLindungi Application. MATRIK : Jurnal Manajemen, Teknik Informatika Dan Rekayasa Komputer, 22(3), 431-442. https://doi.org/https://doi.org/10.30812/matrik.v22i3.2860
Issue
Section
Articles
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
.png)
