Volume 20, Issue 4 (December (Special Issue on ADLEEE) 2024)                   IJEEE 2024, 20(4): 147-161 | Back to browse issues page

XML Print

Download citation:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
Ghasemi E, Razavi S M, Mohamadzadeh S. Performance Comparison of Facial Emotion Recognition: Introducing a Model within the Driver Assistance Framework based on Deep Learning with LBP Feature Extraction for In-Vehicle Applications. IJEEE 2024; 20 (4) :147-161
URL: http://ijeee.iust.ac.ir/article-1-3486-en.html
Performance Comparison of Facial Emotion Recognition: Introducing a Model within the Driver Assistance Framework based on Deep Learning with LBP Feature Extraction for In-Vehicle Applications
Ehsan Ghasemi , Seyyed Mohammad Razavi , Sajad Mohamadzadeh
Abstract:   (124 Views)
This study proposes a descriptor-based approach combined with deep learning, which recognizes facial emotions for safe driving.  Paying attention to the driver's facial expressions is crucial to address the increasing road accidents. This project aims to develop a Facial Emotion Recognition (FER) system that monitors the driver's facial expressions to identify emotions and provide instant assistance for safety control. In the initial stage, Viola-Jones face detection was employed to detect the facial region, followed by Butterworth high-pass filtering to enhance the identified region for locating the eye, nose, and mouth regions, using Viola-Jones face detection. Secondly, the Local Binary Patterns (LBP) feature descriptor is utilized to extract features from the identified eye, nose, and mouth regions. Using 3 RGB channels, the extracted features from these three regions are fed into RessNet-50 and EfficientNet deep networks. The outputs of the two deep learning models' classifiers are combined and integrated using two ensemble methods: ensemble maximum voting and ensemble mean. Based on these combining classifier rules, the performance was evaluated on the JAFFE and KMU-FED databases. The experimental results demonstrate that the proposed method can effectively and with higher accuracy than other competitors recognize emotions in the JAFFE and KMU-FED datasets. The novelty and originality of this paper lie in its significant application in the automotive industry. Implementing our proposed method in a system capable of high accuracy and precision can help mitigate numerous driving hazards. Our approach has achieved 99% and 98% accuracy on the JAFFE and KMU-FED databases, respectively. This high level of accuracy, coupled with its practical relevance, underscores the innovative nature of our work.
Keywords: Ensemble deep learning, combination of classifiers, Driver assistant, Face emotion recognition, Local binary pattern
Full-Text [PDF 1005 kb]   (50 Downloads)    
Type of Study: Research Paper | Subject: Image Processing
Received: 2024/10/07 | Revised: 2025/01/03 | Accepted: 2024/12/28
Rights and permissions
Creative Commons License This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Creative Commons License
© 2022 by the authors. Licensee IUST, Tehran, Iran. This is an open access journal distributed under the terms and conditions of the Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0) license.