Real-Time Driver Drowsiness Detection System Using Dlib based on Driver Eye/Mouth Monitoring Technology
DOI:
https://doi.org/10.26713/cma.v13i2.2034Keywords:
Drowsiness, CNN, IoT, DlibAbstract
The driver sleepiness is common and one of the main causes of road accidents. So, there is a need for automatically detecting of this human behaviour. In any case, if the drivers feel drowsy, they still keep on driving the vehicle, and accidents occur. This study can be implemented using the CNN training model and initiating an alarm if the drowsiness condition is detected. Many of the authors suggested the process of detecting the Drowsiness (Problem Statement) of the drivers using technologies like the Internet of Things (IoT), Deep learning, and Haar cascade (to detect the coordinates of eyes and mouth, which are the target objects). However, this study contributes towards providing the real-time application in co-operating the CNN model and Dlib. Hence, this study proposes a novel embedded system with CNN technology. The CNN model is fed with inputs based on four (4) images related to eye and mouth openings and closings. This application is trained using the CNN model, which takes inputs as images and processes by identifying the features on the face using the Dlib library while representing the change in the state of coordinates of eyes and mouth as Yawning. This approach is achieved using Convolution Neural Network (CNN), pillow, Pygame, OpenCV, and the Dlib, along with providing an alarm when the position of mouth changes. The model is recorded with a maximum validation accuracy of 98% with the minimum recorded loss of less than 0.04% as areal-time application.
Downloads
References
T.J. Arnedt, J. Owens, M. Crouch, J. Stahl and M.A. Carskadon, Neurobehavioral performance of residents after heavy night call vs after alcohol ingestion, Jama 294(9) (2005), 1025 – 1033, DOI: 10.1001/jama.294.9.1025.
A. Bhardwaj, W. Di and J. Wei, Deep Learning Essentials: Your Hands-on Guide to the Fundamentals of Deep Learning and Neural Network Modeling, Packt Publishing Ltd. (2018).
X. Chai, X. Fu, Z. Gan, Y. Lu and Y. Chen, A color image cryptosystem based on dynamic DNA encryption and chaos, Signal Processing 155 (2019), 44 – 62, DOI: 10.1016/j.sigpro.2018.09.029.
V.R.R. Chirra, S. Reddy Uyyala and V.K.K. Kolli, Deep CNN: A machine learning approach for driver drowsiness detection based on eye state, Rev. d’Intelligence Artif. 33(6) (2019), 461 – 466, DOI: 10.18280/ria.330609.
J.P. Cunningham and M.Y. Byron, Dimensionality reduction for large-scale neural recordings, Nature Neuroscience 17(11) (2014), 1500 – 1509, DOI: 10.1038/nn.3776.
E.A. Curran and M.J. Stokes, Learning to control brain activity: A review of the production and control of EEG components for driving brain-computer interface (BCI) systems, Brain and Recognition 51(3) (2003), 326 – 336, DOI: 10.1016/S0278-2626(03)00036-8.
T.A. Dingus, S.K. Jahns, A.D. Horowitz and R. Knipling, Human factors design issues for crash avoidance systems, Human Factors in Intelligent Transportation Systems (1998), 55 – 93.
M. Dua, R. Singla, S. Raj and A. Jangra, Deep CNN models-based ensemble approach to driver drowsiness detection, Neural Computing and Applications 33(8) (2021), 3155 – 3168, DOI: 10.1007/s00521-020-05209-7.
H. Ghassemian, A review of remote sensing image fusion methods, Information Fusion 32 (2016), 75 – 89, DOI: 10.1016/j.inffus.2016.03.003.
R. Jabbar, K. Al-Khalifa, M. Kharbeche, W. Alhajyaseen, M. Jafari and S. Jiang, Real-time driver drowsiness detection for android application using deep neural networks techniques, Procedia Computer Science 130 (2018), 400 – 407, DOI: 10.1016/j.procs.2018.04.060.
S.M. Jameel, M.A. Hashmani, M. Rehman and A. Budiman, Adaptive CNN ensemble for complex multispectral image analysis, Complexity 2020 (2020), DOI: 10.1155/2020/8361989.
M.W. Johns, A sleep physiologist’s view of the drowsy driver, Transportation Research Part F: Traffic Psychology and Behaviour 3(4) (2000), 241 – 249, DOI: 10.1016/S1369-8478(01)00008-0.
S. Kaplan, M.A. Guvensan, A.G. Yavuz and Y. Karalurt, Driver behavior analysis for safe driving: A survey, IEEE Transactions on Intelligent Transportation Systems 16(6) (2015), 3017 – 3032, DOI: 10.1109/TITS.2015.2462084.
A. Kar and P. Corcoran, A review and analysis of eye-gaze estimation systems, algorithms and performance evaluation methods in consumer platforms, IEEE Access 5 (2017), 16495 – 16519, DOI: 10.1109/ACCESS.2017.2735633.
O. Khunpisuth, T. Chotchinasri, V. Koschakosai and N. Hnoohom, Driver drowsiness detection using eye-closeness detection, In: IEEE 12th International Conference on Signal-Image Technology & Internet-Based Systems (SITIS), 2016, pp. 661 – 668, DOI: 10.1109/SITIS.2016.110.
E. Lachat, H. Macher, T. Landes and P. Grussenmeyer, Assessment and calibration of a RGB-D camera (Kinect v2 Sensor) towards a potential use for close-range 3D modeling, Remote Sensing 7(10) (2015), 13070 – 13097, DOI: 10.3390/rs71013070.
B. Lantz, Machine Learning With R: Expert Techniques for Predictive Modeling, Packt Publishing Ltd. (2019).
D.L. Lau and G.R. Arce, Modern Digital Halftoning, Vol. 1, CRC Press, Boca Raton (2018), DOI: 10.1201/9781315219790.
S. Mehta, S. Dadhich, S. Gumber and A.J. Bhatt, Real-time driver drowsiness detection system using eye aspect ratio and eye closure ratio, In: Proceedings of International Conference on Sustainable Computing in Science, Technology and Management (SUSCOM), Amity University Rajasthan, Jaipur, India (2019), DOI: 10.2139/ssrn.3356401.
R. Norman, R. Matzopoulos, P. Groenewald and D. Bradshaw, The high burden of injuries in South Africa, Bulletin of the World Health Organization 85 (2007), 695 – 702, DOI: 10.2471/BLT.06.037184.
M. Ramzan, H.U. Khan, S.M. Awan, A. Ismail, M. Ilyas and A. Mahmood, A survey on state-of-the-art drowsiness detection techniques, IEEE Access 7 (2019), 61904 – 6191, DOI: 10.1109/ACCESS.2019.2914373.
B. Reddy, Y.H. Kim, S. Yun, C. Seo and J. Jang, Real-time driver drowsiness detection for embedded system using model compression of deep neural networks, In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2017, pp. 438 – 445, DOI: 10.1109/CVPRW.2017.59.
V. Sharath, N. Meghana, M. Nayaz, S. Shivakumar and G.L. Sunil, Driver drowsiness detection using haarcascade algorithm, International Journal of Research in Engineering, Science and Management 3(8) (2020), 59 – 60.
C. Shorten and T.M. Khoshgoftaar, A survey on image data augmentation for deep learning, Journal of Big Data 6(1) (2019), Article number: 60, DOI: 10.1186/s40537-019-0197-0.
M. Staubach, Factors correlated with traffic accidents as a basis for evaluating Advanced Driver Assistance Systems, Accident Analysis & Prevention 41(5) (2009), 1025 – 1033, DOI: 10.1016/j.aap.2009.06.014.
C.A. Taylor, J.M. Bell, M.J. Breiding and L. Xu, Traumatic brain injury-related emergency department visits, hospitalizations, and deaths — United States, 2007 and 2013, MMWR Surveillance Summaries 66(9) (2017), 1, DOI: 10.15585/mmwr.ss6609a1.
W.B. Verwey and D.M. Zaidel, Predicting drowsiness accidents from personal attributes, eye blinks and ongoing driving behaviour, Personality and Individual Differences 28(1) (2000), 123 – 142, DOI: 10.1016/S0191-8869(99)00089-6.
C. Ware, Information Visualization: Perception for Design, Morgan Kaufmann (2019).
World Health Organization (WHO), Global Status Report on Road Safety 2015 (2015), URL: http://www.who.int/violence_injury_prevention/road_safety_status/2013/en/index.html.
X. Wu, D. Sahoo and S.C. Hoi, Recent advances in deep learning for object detection, Neurocomputing 396 (2020), 39 – 64, DOI: 10.1016/j.neucom.2020.01.085.
R. Yamashita, M. Nishio, R.K.G. Do and K. Togashi, Convolutional neural networks: an overview and application in radiology, Insights Into Imaging 9(4) (2018), 611 – 629, DOI: 10.1007/s13244-018-0639-9.
C.W. You, N.D. Lane, F. Chen, R. Wang, Z. Chen, T.J. Bao, M. Montes-de-Oca, Y. Cheng, M. Lin, L. Torresani and A.T. Campbell, Carsafe app: Alerting drowsy and distracted drivers using dual cameras on smartphones, In: Proceeding of the 11th Annual International Conference on Mobile Systems, Applications, and Services, 2013, pp. 13 – 26, DOI: 10.1145/2462456.2465428.
Z.Q. Zhao, P. Zheng, S.T. Xu and X. Wu, Object detection with deep learning: A review, IEEE Transactions on Neural Networks and Learning Systems 30(11) (2019), 3212 – 3232, DOI: 110.1109/TNNLS.2018.2876865.
Y. Zhu and N. Zabaras, Bayesian deep convolutional encoder-decoder networks for surrogate modeling and uncertainty quantification, Journal of Computational Physics 366 (2018), 415 – 447, DOI: 10.1016/j.jcp.2018.04.018.
Downloads
Published
How to Cite
Issue
Section
License
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a CCAL that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work.
