Improving Performance Convolutional Neural Networks Using Modified Pooling Function

Achmad Lukman; Wahju Tjahjo Saputro; Erni Seniwati

doi:10.30812/matrik.v23i2.3763

Achmad Lukman Telkom university, Bandung, Indonesia
Wahju Tjahjo Saputro Universitas Muhammadiyah Purworejo, Purworejo, Indonesia
Erni Seniwati Universitas Amikom Yogyakarta, Yogyakarta, Indonesia

DOI: https://doi.org/10.30812/matrik.v23i2.3763

Keywords: Convolutional Neural Network, Visual Geometry Group-16, Qmax pooling function, Qavg pooling function

Abstract

The Visual Geometry Group-16 (VGG16) network architecture, as part of the development of convolutional neural networks, has been popular among researchers in solving classification tasks, so in this paper, we investigated the number of layers to find better performance. In addition, we also proposed two pooling function techniques inspired by existing research on mixed pooling functions, namely Qmax and Qavg. The purpose of the research was to see the advantages of our method; we conducted several test scenarios, including comparing several modified network configurations based on VGG16 as a baseline and involving our pooling technique and existing pooling functions. Then, the results of the first scenario, we selected a network that can adapt well to our pooling technique, which
was then carried out several tests involving the Cifar10, Cifar100, TinyImageNet, and Street View House Numbers (SVHN) datasets as benchmarks. In addition, we were also involved in several existing methods. The experiment results showed that Net-E has the highest performance, with 93.90% accuracy for Cifar10, 71.17% for Cifar100, and 52.84% for TinyImageNet. Still, the accuracy was low when the SVHN dataset was used. In addition, in comparison tests with several optimization algorithms using the Qavg pooling function, it can be seen that the best accuracy results lie in the SGD optimization algorithm, with 89.76% for Cifar10 and 89.06% for Cifar100.

Downloads

Download data is not yet available.

Author Biographies

Wahju Tjahjo Saputro, Universitas Muhammadiyah Purworejo, Purworejo, Indonesia

Wahju Tjahjo Saputro, received the B.Sc. degree in Information system from Universitas Teknologi Digital Indonesia (UTDI) Yogyakarta, Indonesia, in 1996, and the master’s degree in computer science from University of Gadjahmada, Yogyakarta, Indonesia, in 2012. He is currently a Lecturer and Researchers, Department of Information Technology, Universitas Muhammadiyah Purworejo. His current research interests include database, internet of things and data mining

Erni Seniwati, Universitas Amikom Yogyakarta, Yogyakarta, Indonesia

Erni Seniwati received the B.Sc. and the master’s degree in computer science from University of Gadjahmada, Yogyakarta, Indonesia, in 2006 and 2014, respectively. Her research interests are in Fuzzy Logic, Modular Neural Networks, decision support system, Neuro-Fuzzy and Genetic-Fuzzy hybrid approaches., She is currently a Lecturer and Researchers, Department of Information system, Universitas Amikom Yogyakarta. Her current research interests include Fuzzy Logic, AI and data mining

References

[1] X. Ding, G. Ding, Y. Guo, and J. Han, “Centripetal SGD for Pruning Very Deep Convolutional Networks With Complicated
Structure,” in 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), vol. June. IEEE, jun 2019,
pp. 4938–4948, https://doi.org/10.1109/CVPR.2019.00508.
[2] X. Soria, E. Riba, and A. Sappa, “Dense Extreme Inception Network: Towards a Robust CNN Model for Edge Detection,”
in 2020 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE, mar 2020, pp. 1912–1921, https:
//doi.org/10.1109/WACV45572.2020.9093290.
[3] J. Hemalatha, S. Roseline, S. Geetha, S. Kadry, and R. Damaˇseviˇcius, “An Efficient DenseNet-Based Deep Learning Model for
Malware Detection,” Entropy, vol. 23, no. 3, pp. 1–23, mar 2021, https://doi.org/10.3390/e23030344.
[4] F. He, T. Liu, and D. Tao, “Why ResNet Works? Residuals Generalize,” IEEE Transactions on Neural Networks and Learning
Systems, vol. 31, no. 12, pp. 5349–5362, dec 2020, https://doi.org/10.1109/TNNLS.2020.2966319.
[5] C. Wei, S. Kakade, and T. Ma, “The implicit and explicit regularization effects of dropout,” in 37th International Conference
on Machine Learning, ICML 2020, 2020, pp. 10 181–10 192.
[6] X. Liang, L.Wu, J. Li, Y.Wang, Q. Meng, T. Qin,W. Chen, M. Zhang, and T. Y. Liu, “R-Drop: Regularized Dropout for Neural
Networks,” in Advances in Neural Information Processing Systems, vol. 13, 2021, pp. 1–16.
[7] S. K. Roy, M. E. Paoletti, J. M. Haut, E. M. T. Hendrix, and A. Plaza, “A New Max-Min Convolutional Network for Hyperspectral
Image Classification,” in 2021 11th Workshop on Hyperspectral Imaging and Signal Processing: Evolution in Remote
Sensing (WHISPERS). IEEE, mar 2021, pp. 1–5, https://doi.org/10.1109/WHISPERS52202.2021.9483983.
[8] Q. Zhou, Z. Qu, and C. Cao, “Mixed pooling and richer attention feature fusion for crack detection,” Pattern Recognition
Letters, vol. 145, no. May, pp. 96–102, may 2021, https://doi.org/10.1016/j.patrec.2021.02.005.
[9] I. Rodriguez-Martinez, J. Lafuente, R. H. Santiago, G. P. Dimuro, F. Herrera, and H. Bustince, “Replacing pooling functions
in Convolutional Neural Networks by linear combinations of increasing functions,” Neural Networks, vol. 152, no. August, pp.
380–393, aug 2022, https://doi.org/10.1016/j.neunet.2022.04.028.
[10] A. Zafar, M. Aamir, N. Mohd Nawi, A. Arshad, S. Riaz, A. Alruban, A. K. Dutta, and S. Almotairi, “A Comparison of Pooling
Methods for Convolutional Neural Networks,” Applied Sciences, vol. 12, no. 17, pp. 1–21, aug 2022, https://doi.org/10.3390/
app12178643.
[11] C.-Y. Lee, P. Gallagher, and Z. Tu, “Generalizing Pooling Functions in CNNs: Mixed, Gated, and Tree,” IEEE Transactions
on Pattern Analysis and Machine Intelligence, vol. 40, no. 4, pp. 863–875, apr 2018, https://doi.org/10.1109/TPAMI.2017.
2703082.
[12] H. Yang, J. Ni, J. Gao, Z. Han, and T. Luan, “A novel method for peanut variety identification and classification by Improved
VGG16,” Scientific Reports, vol. 11, no. 1, pp. 1–17, aug 2021, https://doi.org/10.1038/s41598-021-95240-y.
[13] M. S and E. Karthikeyan, “Classification of Image using Deep Neural Networks and SoftMax Classifier with CIFAR datasets,” in
2022 6th International Conference on Intelligent Computing and Control Systems (ICICCS). IEEE, may 2022, pp. 1132–1135,
https://doi.org/10.1109/ICICCS53718.2022.9788359.
[14] Y. Le and X. Yang, “Tiny imagenet visual recognition challenge,” CS 231N, vol. 7, no. 7, pp. 1–6, 2015.
[15] M. R. Islam, D. Massicotte, and W.-P. Zhu, “All-ConvNet: A Lightweight All CNN for Neuromuscular Activity Recognition
Using Instantaneous High-Density Surface EMG Images,” in 2020 IEEE International Instrumentation and Measurement
Technology Conference (I2MTC). IEEE, may 2020, pp. 1–6, https://doi.org/10.1109/I2MTC43012.2020.9129362.
[16] J. Yamanaka, S. Kuwashima, and T. Kurita, “Fast and Accurate Image Super Resolution by Deep CNN with Skip Connection
and Network in Network,” in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence
and Lecture Notes in Bioinformatics), 2017, pp. 217–225, https://doi.org/10.1007/978-3-319-70096-0 23.