Learning Accuracy with Particle Swarm Optimization for Music Genre Classification Using Recurrent Neural Networks

Muhammad Rizki; Arief Hermawan; Donny Avianto

doi:10.30812/matrik.v23i2.3037

Muhammad Rizki Universitas Teknologi Yogyakarta, Yogyakarta, Indonesia http://orcid.org/0000-0002-2552-5634
Arief Hermawan Universitas Teknologi Yogyakarta, Yogyakarta, Indonesia http://orcid.org/0000-0002-7061-8274
Donny Avianto Universitas Teknologi Yogyakarta, Yogyakarta, Indonesia http://orcid.org/0000-0001-5499-5478

DOI: https://doi.org/10.30812/matrik.v23i2.3037

Keywords: Learning Rate, Particle Swarm Optimization, Recurrent Neural Network (RNN)

Abstract

Deep learning has revolutionized many fields, but its success often depends on optimal selection hyperparameters, this research aims to compare two sets of learning rates, namely the learning set rates from previous research and rates optimized for Particle Swarm Optimization. Particle Swarm Optimization is learned by mimicking the collective foraging behavior of a swarm of particles, and repeatedly adjusting to improve performance. The results show that the level of Particle Swarm Optimization is better previous level, achieving the highest accuracy of 0.955 compared to the previous best accuracy level of 0.933. In particular, specific levels generated by Particle Swarm Optimization, for example, 0.00163064, achieving competitive accuracy of 0.942-0.945 with shorter computing time compared to the previous rate. These findings underscore the importance of choosing the right learning rate for optimizing the accuracy of Recurrent Neural Networks and demonstrating the potential of Particle Swarm Optimization to exceed existing research benchmarks. Future work will explore comparative analysis different optimization algorithms to obtain the learning rate and assess their computational efficiency. These further investigations promise to improve the performance optimization of Recurrent Neural Networks goes beyond the limitations of previous research.

Downloads

Download data is not yet available.

References

[1] J. Yang, Y. Li, Q. Liu, L. Li, A. Feng, T. Wang, S. Zheng, A. Xu, and J. Lyu, “Brief introduction of medical database and data
mining technology in big data era,” Journal of Evidence-Based Medicine, vol. 13, no. 1, pp. 57–69, 2020, https://doi.org/10.
1111/jebm.12373.
[2] A. Lewkowycz, Y. Bahri, E. Dyer, J. Sohl-Dickstein, and G. Gur-Ari, “The large learning rate phase of deep learning: the
catapult mechanism,” arXiv:2003.02218v1, no. 4 Mar, 2020.
[3] Y. Ming, X. Meng, C. Fan, and H. Yu, “Deep learning for monocular depth estimation: A review,” Neurocomputing, vol. 438,
pp. 14–33, 2021, https://doi.org/10.1016/j.neucom.2020.12.089.
[4] S. Xie, Z. Yu, and Z. Lv, “Multi-disease prediction based on deep learning: A survey,” CMES - Computer Modeling in Engineering
and Sciences, vol. 127, no. 3, 2021, https://doi.org/10.32604/CMES.2021.016728.
[5] S. Minaee, N. Kalchbrenner, E. Cambria, N. Nikzad, M. Chenaghlu, and J. Gao, “Deep Learning-based Text Classification: A
Comprehensive Review,” ACM Computing Surveys, vol. 54, no. 3, pp. 1–40, apr 2022, https://doi.org/10.1145/3439726.
[6] M. R. Karim, O. Beyan, A. Zappa, I. G. Costa, D. Rebholz-Schuhmann, M. Cochez, and S. Decker, “Deep learning-based
clustering approaches for bioinformatics,” Briefings in Bioinformatics, vol. 22, no. 1, pp. 393–415, 2021, https://doi.org/10.
1093/bib/bbz170.
[7] J. Konar, P. Khandelwal, and R. Tripathi, “Comparison of Various Learning Rate Scheduling Techniques on Convolutional
Neural Network,” 2020 IEEE International Students’ Conference on Electrical, Electronics and Computer Science, SCEECS
2020, 2020, https://doi.org/10.1109/SCEECS48394.2020.94.
[8] M. Nagy, M. El-Sersy, M. A. Mohamed, and K. S. Alou, “A Modied Method for Detecting DDoS Attacks Based on Articial
Neural Networks,” The 1st International Conference on Information Technology (IEEE/ITMUSTCONF), 2019.
[9] A. R. Asif, A. Waris, S. O. Gilani, M. Jamil, H. Ashraf, M. Shafique, and I. K. Niazi, “Performance evaluation of convolutional
neural network for hand gesture recognition using EMG,” Sensors (Switzerland), vol. 20, no. 6, pp. 1–11, 2020, https://doi.org/
10.3390/s20061642.
[10] J. Jepkoech, D. M. Mugo, B. K. Kenduiywo, and E. C. Too, “The Effect of Adaptive Learning Rate on the Accuracy of
Neural Networks,” International Journal of Advanced Computer Science and Applications, vol. 12, no. 8, pp. 736–751, 2021,
https://doi.org/10.14569/IJACSA.2021.0120885.
[11] R. Guha, “Benchmarking Gradient Based Optimizers’ Sensitivity to Learning Rate,” SSRN Electronic Journal, no. January, pp.
1–33, 2023.
[12] Y. Aprizal, R. I. Zainal, and A. Afriyudi, “Perbandingan Metode Backpropagation dan Learning Vector Quantization (LVQ)
Dalam Menggali Potensi Mahasiswa Baru di STMIK PalComTech,” MATRIK : Jurnal Manajemen, Teknik Informatika dan
Rekayasa Komputer, vol. 18, no. 2, pp. 294–301, 2019, https://doi.org/10.30812/matrik.v18i2.387.
[13] M. A. Jamal, M. Brown, M. H. Yang, L. Wang, and B. Gong, “Rethinking Class-Balanced Methods for Long-Tailed Visual
Recognition from a Domain Adaptation Perspective,” Proceedings of the IEEE Computer Society Conference on Computer
Vision and Pattern Recognition, pp. 7607–7616, 2020, https://doi.org/10.1109/CVPR42600.2020.00763.
[14] M. Kim and H. Byun, “Learning Texture Invariant Representation for Domain Adaptation of Semantic Segmentation,” Proceedings
of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 12 972–12 981, 2020,
https://doi.org/10.1109/CVPR42600.2020.01299.
[15] F. Mulier and V. Cherkassky, “Learning rate schedules for self-organizing maps,” Proceedings - International Conference on
Pattern Recognition, vol. 2, pp. 224–228, 1994, https://doi.org/10.1109/icpr.1994.576908.
[16] B. Bischl, M. Binder, M. Lang, T. Pielok, J. Richter, S. Coors, J. Thomas, T. Ullmann, M. Becker, A. L. Boulesteix, D. Deng,
and M. Lindauer, “Hyperparameter optimization: Foundations, algorithms, best practices, and open challenges,” Wiley Interdisciplinary
Reviews: Data Mining and Knowledge Discovery, vol. 13, no. 2, pp. 1–43, 2023, https://doi.org/10.1002/widm.1484.
[17] ANDRADA, “GTZAN Dataset - Music Genre Classification,” 2020.
[18] A. Anggrawan, “Application of KNN Machine Learning and Fuzzy C-Means to Diagnose Diabetes,” vol. 22, no. 2, pp. 405–418,
2023, https://doi.org/10.30812/matrik.v22i2.2777.
[19] Z. Allen-Zhu and Y. Li, “Can SGD learn recurrent neural networks with provable generalization?” Advances in Neural Information
Processing Systems, vol. 32, no. NeurIPS, 2019.
[20] M. Jahandideh-Tehrani, O. Bozorg-Haddad, and H. A. Lo´aiciga, “Application of particle swarm optimization to water management:
an introduction and overview,” Environmental Monitoring and Assessment, vol. 192, no. 5, 2020, https://doi.org/10.1007/
s10661-020-8228-z.
[21] T. H. Zhao, M. I. Khan, and Y. M. Chu, “Artificial neural networking (ANN) analysis for heat and entropy generation in flow of
non-Newtonian fluid between two rotating disks,” Mathematical Methods in the Applied Sciences, vol. 46, no. 3, pp. 3012–3030,
2023, https://doi.org/10.1002/mma.7310.
Matrik: