A Video-Based Emotion Recognition Model Based on Facial Expression Dynamics Using a Hybrid CNN–RNN–RBM Network

Abstract

Facial expressions represent one of the most important forms of non-verbal communication, enabling humans to convey emotional states and interact without spoken language. In recent studies on emotion recognition based on facial expression variations, deep learning methods have been widely employed. In this regard, the selection of an appropriate neural network architecture depends on the nature of the input data. In the present study, video data are utilized, and Recurrent Neural Networks (RNN) are applied for Facial Expression Recognition (FER). Furthermore, to achieve improved performance, a hybrid architecture consisting of Convolutional Neural Networks (CNN), RNN, and Restricted Boltzmann Machines (RBM) is employed. Ultimately, this paper proposes a deep learning-based algorithm implemented in Python, relying on a hybrid CNN–LSTM–RBM neural network for emotion recognition from facial expressions in video sequences. The model is designed to classify seven emotional states, namely happiness, sadness, disgust, excitement, contempt, fear, and anger. The CK+48 dataset is used for training and evaluation of the proposed model. The experimental results demonstrate that the proposed neural network achieves a training accuracy of 100% with a training loss of 0.02, indicating the effectiveness and strong performance of the proposed approach.