Drowsiness Detection Based on Yawning Using Modified Pre-trained Model MobileNetV2 and ResNet50
DOI:
https://doi.org/10.30812/matrik.v22i3.2785Keywords:
Drowsiness detection, Facial-expression change, MobileNetV2, ResNet50, Yawn detectionAbstract
Traffic accidents are fatal events that need special attention. According to research by the National Transportation Safety Committee, 80% of traffic accidents are caused by human error, one of which is tired and drowsy drivers. The brain can interpret the vital fatigue of a drowsy driver sign as yawning. Therefore, yawning detection for preventing drowsy drivers’ imprudent can be developed using computer vision. This method is easy to implement and does not affect the driver when handling a vehicle. The research aimed to detect drowsy drivers based on facial expression changes of yawning by combining the Haar Cascade classifier and a modified pre-trained model, MobileNetV2 and ResNet50. Both proposed models accurately detected real-time images using a camera. The analysis showed that the yawning detection model based on the ResNet50 algorithm is more reliable, with the model obtaining 99% of accuracy. Furthermore, ResNet50 demonstrated reproducible outcomes for yawning detection, considering having good training capabilities and overall evaluation results.
Downloads
References
Drowsiness during Medium-Distance Journeys,†Entropy, vol. 23, no. 2, pp. 1–12, feb 2021.
[2] Z. Liu, Y. Peng, and W. Hu, “Driver Fatigue Detection Based on Deeply-Learned Facial Expression Representation,†Journal
of Visual Communication and Image Representation, vol. 71, p. 102723, aug 2020.
[3] Y. Ma, B. Chen, R. Li, C.Wang, J.Wang, Q. She, Z. Luo, and Y. Zhang, “Driving Fatigue Detection from EEG Using a Modified
PCANet Method,†Computational Intelligence and Neuroscience, vol. 2019, pp. 1–9, jul 2019.
[4] Z. Liu,W.Wei, H.Wang, Y. Zhang, Q. Zhang, and S. Li, “Intrusion Detection Based on Parallel Intelligent Optimization Feature
Extraction and Distributed Fuzzy Clustering in WSNs,†IEEE Access, vol. 6, no. November, pp. 72 201–72 211, 2018.
[5] J. Xi, S.Wang, T. Ding, J. Tian, H. Shao, and X. Miao, “Detection Model on Fatigue Driving Behaviors Based on the Operating
Parameters of Freight Vehicles,†Applied Sciences, vol. 11, no. 15, pp. 1–16, aug 2021.
[6] C. B. S. Maior, M. J. d. C. Moura, J. M. M. Santana, and I. D. Lins, “Real-time classification for autonomous drowsiness
detection using eye aspect ratio,†Expert Systems with Applications, vol. 158, p. 113505, nov 2020.
[7] C. Ryan, B. O’Sullivan, A. Elrasad, A. Cahill, J. Lemley, P. Kielty, C. Posch, and E. Perot, “Real-Time Face & Eye Tracking
and Blink Detection Using Event Cameras,†Neural Networks, vol. 141, no. 2021, pp. 87–97, 2021.
[8] H. Zidny Ilmadina, D. Apriliani, and D. S. Wibowo, “Deteksi Pengendara Mengantuk dengan Kombinasi Haar Cascade Classifier
dan Support Vector Machine,†Jurnal Informatika : Jurnal Pengembangan IT (JPIT), vol. 7, no. 1, pp. 1–7, 2022.
[9] C. Jacob´e de Naurois, C. Bourdin, A. Stratulat, E. Diaz, and J. L. Vercher, “Detection and Prediction of Driver Drowsiness
Using Artificial Neural Network Models,†Accident Analysis and Prevention, vol. 126, no. July, pp. 95–104, 2019.
[10] S. Rajamohana, E. Radhika, S. Priya, and S. Sangeetha, “Driver drowsiness detection system using hybrid approach of convolutional
neural network and bidirectional long short term memory (CNN BILSTM),†Materials Today: Proceedings, vol. 45,
no. 2, pp. 2897–2901, 2021.
[11] M. Knapik and B. Cyganek, “Driver’s fatigue recognition based on yawn detection in thermal images,†Neurocomputing, vol.
338, pp. 274–292, apr 2019.
[12] M. Kim and J. Koo, “Embedded System Performance Analysis for Implementing a Portable Drowsiness Detection System for
Drivers,†Technologies, vol. 11, no. 1, p. 124, dec 2022.
[13] K. Chen, T. Zhu, S. Li, and Y. Shi, “Real-time Yawning Detection Based on Machine Learning Algorithm and Time Series
Classification using Facial Feature Points,†in 2021 International Conference on High Performance Big Data and Intelligent
Systems (HPBD&IS). IEEE, dec 2021, pp. 276–280.
[14] A. Rahman, M. B. H. Hriday, and R. Khan, “Computer Vision-Based Approach to Detect Fatigue Driving and Face Mask for
Edge Computing Device,†Heliyon, vol. 8, no. 10, pp. 1–16, oct 2022.
[15] D. Vazquez, “Yawn Dataset— Kaggle.†[Online]. Available: https://www.kaggle.com/datasets/davidvazquezcic/yawn-dataset
[16] R. Raj, P. Rajiv, P. Kumar, M. Khari, E. Verd´u, R. G. Crespo, and G. Manogaran, “Feature based video stabilization based on
boosted HAAR Cascade and representative point matching algorithm,†Image and Vision Computing, vol. 101, p. 103957, sep
2020.
[17] D. Yang, B. Peng, Z. Al-Huda, A. Malik, and D. Zhai, “An Overview of Edge and Object Contour Detection,†Neurocomputing,
vol. 488, no. June, pp. 470–493, jun 2022.
[18] M. Sandler, A. Howard, M. Zhu, and A. Zhmoginov, “MobileNetV2: Inverted Residuals and Linear Bottlenecks,†in Proceedings
of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR),, 2018, pp. 4510–4520.
[19] K. He, X. Zhang, S. Ren, and J. Sun, “Deep Residual Learning for Image Recognition,†in 2016 IEEE Conference on Computer
Vision and Pattern Recognition (CVPR), 2016, pp. 770–778.
[20] A. Michele, V. Colin, and D. D. Santika, “MobileNet Convolutional Neural Networks and Support Vector Machines for Palmprint
Recognition,†Procedia Computer Science, vol. 157, pp. 110–117, 2019.
[21] M. H. Rahman, M. K. A. Jannat, M. S. Islam, G. Grossi, S. Bursic, and M. Aktaruzzaman, “Real-time face mask position
recognition system based on MobileNet model,†Smart Health, vol. 28, no. June, pp. 1–12, jun 2023.
[22] I. Shafi, A. Mazahir, A. Fatima, and I. Ashraf, “Internal Defects Detection and Classification in Hollow Cylindrical Surfaces
Using Single Shot Detection and MobileNet,†Measurement, vol. 202, p. 111836, oct 2022.
[23] W. Qi, “Object detection in high resolution optical image based on deep learning technique,†Natural Hazards Research, vol. 2,
no. 4, pp. 384–392, dec 2022.
[24] D. Sarwinda, R. H. Paradisa, A. Bustamam, and P. Anggia, “Deep Learning in Image Classification using Residual Network
(ResNet) Variants for Detection of Colorectal Cancer,†Procedia Computer Science, vol. 179, pp. 423–431, 2021.
[25] J. Chai, H. Zeng, A. Li, and E. W. Ngai, “Deep Learning in Computer Vision: A Critical Review of Emerging Techniques and
Application Scenarios,†Machine Learning with Applications, vol. 6, p. 100134, dec 2021.
[26] C. H. Karadal, M. C. Kaya, T. Tuncer, S. Dogan, and U. R. Acharya, “Automated classification of remote sensing images using
multileveled MobileNetV2 and DWT techniques,†Expert Systems with Applications, vol. 185, pp. 1–12, dec 2021.
Downloads
Published
Issue
Section
How to Cite
Similar Articles
- Muhammad Amirul Mukminin, Tio Dharmawan, Muhamad Arief Hidayat, Gender Classification Using Viola Jones, Orthogonal Difference Local Binary Pattern and Principal Component Analysis , MATRIK : Jurnal Manajemen, Teknik Informatika dan Rekayasa Komputer: Vol. 23 No. 3 (2024)
- Imanuddin Imanuddin, Fachrid Alhadi, Raza Oktafian, Ahmad Ihsan, Deteksi Mata Mengantuk pada Pengemudi Mobil Menggunakan Metode Viola Jones , MATRIK : Jurnal Manajemen, Teknik Informatika dan Rekayasa Komputer: Vol. 18 No. 2 (2019)
- Nella Rosa Sudianjaya, Chastine Fatichah, Segmentation and Classification of Breast Cancer Histopathological Image Utilizing U-Net and Transfer Learning ResNet50 , MATRIK : Jurnal Manajemen, Teknik Informatika dan Rekayasa Komputer: Vol. 24 No. 1 (2024)
- Bambang Krismono Triwijoyo, Ahmat Adil, Anthony Anggrawan, Convolutional Neural Network With Batch Normalization for Classification of Emotional Expressions Based on Facial Images , MATRIK : Jurnal Manajemen, Teknik Informatika dan Rekayasa Komputer: Vol. 21 No. 1 (2021)
- Ervina Farijki, Bambang Krismono Triwijoyo, SEGMENTASI CITRA MRI MENGGUNAKAN DETEKSI TEPI UNTUK IDENTIFIKASI KANKER PAYUDARA , MATRIK : Jurnal Manajemen, Teknik Informatika dan Rekayasa Komputer: Vol. 15 No. 2 (2016)
- Radimas Putra Muhammad Davi Labib, Sirojul Hadi, Parama Diptya Widayaka, Low Cost System for Face Mask Detection Based Haar Cascade Classifier Method , MATRIK : Jurnal Manajemen, Teknik Informatika dan Rekayasa Komputer: Vol. 21 No. 1 (2021)
- Didih Rizki Chandranegara, Faras Haidar Pratama, Sidiq Fajrianur, Moch Rizky Eka Putra, Zamah Sari, Automated Detection of Breast Cancer Histopathology Image Using Convolutional Neural Network and Transfer Learning , MATRIK : Jurnal Manajemen, Teknik Informatika dan Rekayasa Komputer: Vol. 22 No. 3 (2023)
- Muhammad Alkaff, Muhammad Afrizal Miqdad, Muhammad Fachrurrazi, Muhammad Nur Abdi, Ahmad Zainul Abidin, Raisa Amalia, Hate Speech Detection for Banjarese Languages on Instagram Using Machine Learning Methods , MATRIK : Jurnal Manajemen, Teknik Informatika dan Rekayasa Komputer: Vol. 22 No. 3 (2023)
- Ardi Mardiana, Ade Bastian, Ano Tarsono, Dony Susandi, Safari Yonasi, Optimized YOLOv8 Model for Accurate Detection and Quantificationof Mango Flowers , MATRIK : Jurnal Manajemen, Teknik Informatika dan Rekayasa Komputer: Vol. 24 No. 3 (2025)
- Imam Fahrur Rozi, Ahmadi Yuli Ananta, Endah Septa Sintiya, Astrifidha Rahma Amalia, Yuri Ariyanto, Arin Kistia Nugraeni, Analyzing the Application of Optical Character Recognition: A Case Study in International Standard Book Number Detection , MATRIK : Jurnal Manajemen, Teknik Informatika dan Rekayasa Komputer: Vol. 24 No. 2 (2025)
You may also start an advanced similarity search for this article.
Most read articles by the same author(s)
- Dairoh Dairoh, Very Kurnia Bakti, Muhammad Naufal, Neural Network dan Particle Swam Optimization untuk Penunjang Keputusan Antipasi Mahasiswa Pra Lulus Bekerja Sesuai Bidang , MATRIK : Jurnal Manajemen, Teknik Informatika dan Rekayasa Komputer: Vol. 21 No. 1 (2021)