The Mitigating Overfitting in Sentiment Analysis Insights from CNN-LSTM Hybrid Models
Keywords:
Convolutional Neural Networks, Hybrid Models, Long Short-Term Memory, Mitigating Overfitting, Sentiment Analysis
Abstract
This study aims to improve sentiment analysis accuracy and address overfitting challenges in deep learning models by developing a hybrid model based on Convolutional Neural Networks and Long Short-Term Memory Networks. The research methodology involved multiple stages, starting with preprocessing a dataset of 5,456 rows. This process included removing duplicate data, empty entries, and neutral sentiments, resulting in 2,685 usable rows. To overcome data quantity limitations, data augmentation expanded the training dataset from 2,148 to 10,740 samples. Data transformation was carried out using tokenization, padding, and embedding techniques, leveraging Word2Vec and GloVe to produce numerical representations of textual data. The hybrid model demonstrated strong performance, achieving a training accuracy of 99.51%, validation accuracy of 99.25%, and testing accuracy of 87.34%, with a loss value of 0.56. Evaluation metrics showed precision, recall, and F1-Score values of 86%, 87%, and 86%, respectively. The hybrid model outperformed individual models, including Convolutional Neural Networks (70% accuracy) and Long Short-Term Memory Networks (81% accuracy). It also surpassed other hybrid models, such as the multiscale Convolutional Neural Network-Long Short-Term Memory Network, which achieved a maximum accuracy of 89.25%. The implications of this study demonstrate that the hybrid model based on Convolutional Neural Networks and Long Short-Term Memory Networks effectively improves sentiment analysis accuracy while reducing the risk of overfitting, particularly in small or imbalanced datasets. Future research is recommended to enhance data quality, adopt more advanced embedding techniques, and optimize model configurations to achieve better performance.
Downloads
Download data is not yet available.
References
[1] J. Khan, N. Ahmad, S. Khalid, F. Ali, and Y. Lee, “Sentiment and Context-Aware Hybrid DNN With Attention for Text Sentiment
Classification,” IEEE Access, vol. 11, no. 3, pp. 28 162–28 179, 2023, https://doi.org/10.1109/ACCESS.2023.3259107.
[2] N. A. Semary, W. Ahmed, K. Amin, P. Pławiak, and M. Hammad, “Improving sentiment classification using a RoBERTa-based
hybrid model,” Frontiers in Human Neuroscience, vol. 17, no. 12, pp. 1–10, 2023, https://doi.org/10.3389/fnhum.2023.1292010.
[3] M. S. Islam, M. N. Kabir, N. A. Ghani, K. Z. Zamli, N. S. A. Zulkifli, M. M. Rahman, and M. A. Moni, “”Challenges and future
in deep learning for sentiment analysis: a comprehensive review and a proposed novel hybrid approach”,” Artificial Intelligence
Review, vol. 57, no. 3, pp. 1–79, 2024, https://doi.org/10.1007/s10462-023-10651-9.
[4] R. Geethanjali and A. Valarmathi, “A novel hybrid deep learning IChOA-CNN-LSTM model for modality-enriched and multilingual
emotion recognition in social media,” Scientific reports, vol. 14, no. 1, p. 22270, 2024, https://doi.org/10.1038/
s41598-024-73452-2.
[5] A. Wahdan, S. Hantoobi, S. A. Salloum, and K. Shaalan, “A systematic review of text classification research based on deep
learning models in Arabic language,” International Journal of Electrical and Computer Engineering, vol. 10, no. 6, pp. 6629–
6643, 2020, https://doi.org/10.11591/IJECE.V10I6.PP6629-6643.
[6] K. M. Hasib, S. Azam, A. Karim, A. A. Marouf, F. M. M. Shamrat, S. Montaha, K. C. Yeo, M. Jonkman, R. Alhajj, and J. G.
Rokne, “MCNN-LSTM: Combining CNN and LSTM to Classify Multi-Class Text in Imbalanced News Data,” IEEE Access,
vol. 11, no. 9, pp. 93 048–93 063, 2023, https://doi.org/10.1109/ACCESS.2023.3309697.
[7] N. Zhang, J. Xiong, Z. Zhao, M. Feng, X. Wang, Y. Qiao, and C. Jiang, “Dose My Opinion Count? A CNN-LSTM Approach
for Sentiment Analysis of Indian General Elections,” Journal of Theory and Practice of Engineering Science, vol. 4, no. 05, pp.
40–50, 2024, https://doi.org/10.53469/jtpes.2024.04(05).06.
[8] A. Sungheetha, “TransCapsule Model for Sentiment Classification,” Journal of Artificial Intelligence and Capsule Networks,
vol. 02, no. 03, pp. 163–169, 2020, https://doi.org/10.36548/jaicn.2020.3.003.
[9] D. Chai, W. Wu, Q. Han, W. Fei, and J. Li, “Description based text classification with reinforcement learning,” in 37th International
Conference on Machine Learning, ICML 2020, vol. 119, no. 1, 2020, pp. 1348–1359.
[10] P. Sudhir and V. Deshakulkarni, “Comparative study of various approaches , applications and classifiers for sentiment analysis,”
Global Transitions Proceedings, vol. 2, no. 2, pp. 205–211, 2021, https://doi.org/10.1016/j.gltp.2021.08.004.
[11] L. Irfan, S. Hussain, M. Ayoub, Y. Yu, and A. Khan, “A Comparative Analysis of Social Communication Applications using
Aspect Based Sentiment Analysis,” Pakistan Journal of Engineering and Technology, PakJET, vol. 5, no. 3, pp. 44–50, 2022,
https://doi.org/10.51846/vol5iss3pp44-50.
[12] O. Iparraguirre-villanueva, A. Alvarez-risco, J. Luis, H. Salazar, S. Beltozar-clemente, J. Zapata-paulini, A. Y. Jaime, and
M. Cabanillas-carbonell, “The Public Health Contribution of Sentiment Analysis of Monkeypox Tweets to Detect Polarities
Using the CNN-LSTM Model number,” vaccines, vol. 11, no. 312, pp. 1–12, 2023, https://doi.org/10.3390/vaccines11020312.
[13] A. Gupta and P. Agarwal, “Integrating CRM and ERP Insights for Optimized Product Development Using CNN-LSTM Hybrid
Models,” International Journal of Computer Trends and Technology, vol. 72, no. 8, pp. 91–97, 2024, https://doi.org/10.14445/
22312803/IJCTT-V72I8P113.
[14] Y. Zhou, Q. Zhang, D. Wang, and X. Gu, “Text Sentiment Analysis Based on a New Hybrid Network Model,” p. 6774320,
2022, https://doi.org/10.1155/2022/6774320.
[15] N. Jin, J. Wu, X. Ma, K. Yan, and Y. Mo, “Multi-task learning model based on Multi-scale CNN and LSTM for sentiment
classification,” IEEE Access, vol. 8, no. 4, pp. 77 060–77 072, 2020, https://doi.org/10.1109/ACCESS.2020.2989428.
[16] S. Soumya and K. V. Pramod, “Hybrid Deep Learning Approach for Sentiment Classification of Malayalam Tweets,” (IJACSA)
International Journal of Advanced Computer Science and Applications, vol. 13, no. 4, pp. 891–899, 2022, https://doi.org/10.
14569/IJACSA.2022.01304103.
[17] A. Mohta, A. Jain, A. Saluja, and S. Dahiya, “Pre-processing and emoji classification of whatsapp chats for sentiment analysis,”
in Proceedings of the 4th International Conference on IoT in Social, Mobile, Analytics and Cloud, ISMAC 2020, 2020, pp.
514–519, https://doi.org/10.1109/I-SMAC49090.2020.9243443.
[18] S. Wu, K. Roberts, S. Datta, J. Du, Z. Ji, Y. Si, S. Soni, Q. Wang, Q. Wei, Y. Xiang, B. Zhao, and H. Xu, “Deep learning in
clinical natural language processing: A methodical review,” pp. 457–470, 2020, https://doi.org/10.1093/jamia/ocz200.
[19] L. Cabral, J. Monteiro, J. Franco da Silva, C. Mattos, and P. Mour˜ao, “FakeWhastApp.BR: NLP and Machine Learning Techniques
for Misinformation Detection in Brazilian Portuguese WhatsApp Messages,” in Proceedings of the 23rd International
Conference on Enterprise Information Systems, vol. 1, 2021, pp. 63–74, https://doi.org/10.5220/0010446800630074.
[20] P. K. Jain, V. Saravanan, and R. Pamula, “A Hybrid CNN-LSTM : A Deep Learning Approach for Consumer Sentiment Analysis
Using Qualitative User-Generated Contents,” vol. 20, no. 5, pp. 1–15, 2021, https://doi.org/10.1145/3457206.
[21] J. Sun, R. Jin, X. Ma, J.-y. Park, K.-a. Sohn, and T.-s. Chung, “Gated Convolutional Neural Networks for Text Classification,”
in Advances in Computer Science and Ubiquitous Computing, J. J. Park, S. J. Fong, Y. Pan, and Y. Sung, Eds. Singapore:
Springer Singapore, 2021, pp. 309–316, https://doi.org/10.1007/978-981-15-9343-7 43.
[22] Y. Zhou, Q. Zhang, D. Wang, and X. Gu, “Text Sentiment Analysis Based on a New Hybrid Network Model,” Computational
Intelligence and Neuroscience, vol. 2022, no. 12, pp. 1–15, 2022, https://doi.org/10.1155/2022/6774320.
[23] L. Khan, A. Amjad, K. M. Afaq, and H.-t. Chang, “Deep Sentiment Analysis Using CNN-LSTM Architecture of English
and Roman Urdu Text Shared in Social Media,” Applied Sciences, vol. 12, no. 6, pp. 1–18, 2022, https://doi.org/10.3390/
app12052694.
[24] S. Riyadi, A. Divayu Andriyani, and S. Noraini Sulaiman, “Improving Hate Speech Detection Using Double-Layers Hybrid
CNN-RNN Model on Imbalanced Dataset,” IEEE Access, vol. 12, no. 10, pp. 159 660–159 668, 2024, https://doi.org/10.1109/
ACCESS.2024.3487433.
[25] S. Susandri, S. Defit, and M. Tajuddin, “Enhancing Text Sentiment Classification with Hybrid CNN-BiLSTM Model on WhatsApp
Group,” Journal of Advances in Information Technology, vol. 15, no. 3, pp. 355–363, 2024, https://doi.org/10.12720/jait.
15.3.355-363.
[26] G. Chao, J. Liu, M. Wang, and D. Chu, “Data augmentation for sentiment classification with semantic preservation and diversity,”
Knowledge-Based Systems, vol. 280, no. 11, p. 111038, 2023, https://doi.org/10.1016/j.knosys.2023.111038.
[27] J. Chen, Z. Yang, and D. Yang, “MixText: Linguistically-Informed Interpolation of Hidden Space for Semi-Supervised Text
Classification,” Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, vol. 1, no. 1, pp.
2147–2157, 2020, https://doi.org/10.18653/v1/2020.acl-main.194.
Classification,” IEEE Access, vol. 11, no. 3, pp. 28 162–28 179, 2023, https://doi.org/10.1109/ACCESS.2023.3259107.
[2] N. A. Semary, W. Ahmed, K. Amin, P. Pławiak, and M. Hammad, “Improving sentiment classification using a RoBERTa-based
hybrid model,” Frontiers in Human Neuroscience, vol. 17, no. 12, pp. 1–10, 2023, https://doi.org/10.3389/fnhum.2023.1292010.
[3] M. S. Islam, M. N. Kabir, N. A. Ghani, K. Z. Zamli, N. S. A. Zulkifli, M. M. Rahman, and M. A. Moni, “”Challenges and future
in deep learning for sentiment analysis: a comprehensive review and a proposed novel hybrid approach”,” Artificial Intelligence
Review, vol. 57, no. 3, pp. 1–79, 2024, https://doi.org/10.1007/s10462-023-10651-9.
[4] R. Geethanjali and A. Valarmathi, “A novel hybrid deep learning IChOA-CNN-LSTM model for modality-enriched and multilingual
emotion recognition in social media,” Scientific reports, vol. 14, no. 1, p. 22270, 2024, https://doi.org/10.1038/
s41598-024-73452-2.
[5] A. Wahdan, S. Hantoobi, S. A. Salloum, and K. Shaalan, “A systematic review of text classification research based on deep
learning models in Arabic language,” International Journal of Electrical and Computer Engineering, vol. 10, no. 6, pp. 6629–
6643, 2020, https://doi.org/10.11591/IJECE.V10I6.PP6629-6643.
[6] K. M. Hasib, S. Azam, A. Karim, A. A. Marouf, F. M. M. Shamrat, S. Montaha, K. C. Yeo, M. Jonkman, R. Alhajj, and J. G.
Rokne, “MCNN-LSTM: Combining CNN and LSTM to Classify Multi-Class Text in Imbalanced News Data,” IEEE Access,
vol. 11, no. 9, pp. 93 048–93 063, 2023, https://doi.org/10.1109/ACCESS.2023.3309697.
[7] N. Zhang, J. Xiong, Z. Zhao, M. Feng, X. Wang, Y. Qiao, and C. Jiang, “Dose My Opinion Count? A CNN-LSTM Approach
for Sentiment Analysis of Indian General Elections,” Journal of Theory and Practice of Engineering Science, vol. 4, no. 05, pp.
40–50, 2024, https://doi.org/10.53469/jtpes.2024.04(05).06.
[8] A. Sungheetha, “TransCapsule Model for Sentiment Classification,” Journal of Artificial Intelligence and Capsule Networks,
vol. 02, no. 03, pp. 163–169, 2020, https://doi.org/10.36548/jaicn.2020.3.003.
[9] D. Chai, W. Wu, Q. Han, W. Fei, and J. Li, “Description based text classification with reinforcement learning,” in 37th International
Conference on Machine Learning, ICML 2020, vol. 119, no. 1, 2020, pp. 1348–1359.
[10] P. Sudhir and V. Deshakulkarni, “Comparative study of various approaches , applications and classifiers for sentiment analysis,”
Global Transitions Proceedings, vol. 2, no. 2, pp. 205–211, 2021, https://doi.org/10.1016/j.gltp.2021.08.004.
[11] L. Irfan, S. Hussain, M. Ayoub, Y. Yu, and A. Khan, “A Comparative Analysis of Social Communication Applications using
Aspect Based Sentiment Analysis,” Pakistan Journal of Engineering and Technology, PakJET, vol. 5, no. 3, pp. 44–50, 2022,
https://doi.org/10.51846/vol5iss3pp44-50.
[12] O. Iparraguirre-villanueva, A. Alvarez-risco, J. Luis, H. Salazar, S. Beltozar-clemente, J. Zapata-paulini, A. Y. Jaime, and
M. Cabanillas-carbonell, “The Public Health Contribution of Sentiment Analysis of Monkeypox Tweets to Detect Polarities
Using the CNN-LSTM Model number,” vaccines, vol. 11, no. 312, pp. 1–12, 2023, https://doi.org/10.3390/vaccines11020312.
[13] A. Gupta and P. Agarwal, “Integrating CRM and ERP Insights for Optimized Product Development Using CNN-LSTM Hybrid
Models,” International Journal of Computer Trends and Technology, vol. 72, no. 8, pp. 91–97, 2024, https://doi.org/10.14445/
22312803/IJCTT-V72I8P113.
[14] Y. Zhou, Q. Zhang, D. Wang, and X. Gu, “Text Sentiment Analysis Based on a New Hybrid Network Model,” p. 6774320,
2022, https://doi.org/10.1155/2022/6774320.
[15] N. Jin, J. Wu, X. Ma, K. Yan, and Y. Mo, “Multi-task learning model based on Multi-scale CNN and LSTM for sentiment
classification,” IEEE Access, vol. 8, no. 4, pp. 77 060–77 072, 2020, https://doi.org/10.1109/ACCESS.2020.2989428.
[16] S. Soumya and K. V. Pramod, “Hybrid Deep Learning Approach for Sentiment Classification of Malayalam Tweets,” (IJACSA)
International Journal of Advanced Computer Science and Applications, vol. 13, no. 4, pp. 891–899, 2022, https://doi.org/10.
14569/IJACSA.2022.01304103.
[17] A. Mohta, A. Jain, A. Saluja, and S. Dahiya, “Pre-processing and emoji classification of whatsapp chats for sentiment analysis,”
in Proceedings of the 4th International Conference on IoT in Social, Mobile, Analytics and Cloud, ISMAC 2020, 2020, pp.
514–519, https://doi.org/10.1109/I-SMAC49090.2020.9243443.
[18] S. Wu, K. Roberts, S. Datta, J. Du, Z. Ji, Y. Si, S. Soni, Q. Wang, Q. Wei, Y. Xiang, B. Zhao, and H. Xu, “Deep learning in
clinical natural language processing: A methodical review,” pp. 457–470, 2020, https://doi.org/10.1093/jamia/ocz200.
[19] L. Cabral, J. Monteiro, J. Franco da Silva, C. Mattos, and P. Mour˜ao, “FakeWhastApp.BR: NLP and Machine Learning Techniques
for Misinformation Detection in Brazilian Portuguese WhatsApp Messages,” in Proceedings of the 23rd International
Conference on Enterprise Information Systems, vol. 1, 2021, pp. 63–74, https://doi.org/10.5220/0010446800630074.
[20] P. K. Jain, V. Saravanan, and R. Pamula, “A Hybrid CNN-LSTM : A Deep Learning Approach for Consumer Sentiment Analysis
Using Qualitative User-Generated Contents,” vol. 20, no. 5, pp. 1–15, 2021, https://doi.org/10.1145/3457206.
[21] J. Sun, R. Jin, X. Ma, J.-y. Park, K.-a. Sohn, and T.-s. Chung, “Gated Convolutional Neural Networks for Text Classification,”
in Advances in Computer Science and Ubiquitous Computing, J. J. Park, S. J. Fong, Y. Pan, and Y. Sung, Eds. Singapore:
Springer Singapore, 2021, pp. 309–316, https://doi.org/10.1007/978-981-15-9343-7 43.
[22] Y. Zhou, Q. Zhang, D. Wang, and X. Gu, “Text Sentiment Analysis Based on a New Hybrid Network Model,” Computational
Intelligence and Neuroscience, vol. 2022, no. 12, pp. 1–15, 2022, https://doi.org/10.1155/2022/6774320.
[23] L. Khan, A. Amjad, K. M. Afaq, and H.-t. Chang, “Deep Sentiment Analysis Using CNN-LSTM Architecture of English
and Roman Urdu Text Shared in Social Media,” Applied Sciences, vol. 12, no. 6, pp. 1–18, 2022, https://doi.org/10.3390/
app12052694.
[24] S. Riyadi, A. Divayu Andriyani, and S. Noraini Sulaiman, “Improving Hate Speech Detection Using Double-Layers Hybrid
CNN-RNN Model on Imbalanced Dataset,” IEEE Access, vol. 12, no. 10, pp. 159 660–159 668, 2024, https://doi.org/10.1109/
ACCESS.2024.3487433.
[25] S. Susandri, S. Defit, and M. Tajuddin, “Enhancing Text Sentiment Classification with Hybrid CNN-BiLSTM Model on WhatsApp
Group,” Journal of Advances in Information Technology, vol. 15, no. 3, pp. 355–363, 2024, https://doi.org/10.12720/jait.
15.3.355-363.
[26] G. Chao, J. Liu, M. Wang, and D. Chu, “Data augmentation for sentiment classification with semantic preservation and diversity,”
Knowledge-Based Systems, vol. 280, no. 11, p. 111038, 2023, https://doi.org/10.1016/j.knosys.2023.111038.
[27] J. Chen, Z. Yang, and D. Yang, “MixText: Linguistically-Informed Interpolation of Hidden Space for Semi-Supervised Text
Classification,” Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, vol. 1, no. 1, pp.
2147–2157, 2020, https://doi.org/10.18653/v1/2020.acl-main.194.
Published
2025-03-10
How to Cite
Susandri, S., Zamsuri, A., Nasution, N., Efendi, Y., & Alwan, H. (2025). The Mitigating Overfitting in Sentiment Analysis Insights from CNN-LSTM Hybrid Models. MATRIK : Jurnal Manajemen, Teknik Informatika Dan Rekayasa Komputer, 24(2), 297-308. https://doi.org/https://doi.org/10.30812/matrik.v24i2.4742
Issue
Section
Articles
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
.png)
