Facial expression recognition in videos

Abstract

Facial expression recognition is considered as the most effective, naturally distinguished way for humans to communicate emotions, express their feelings, to clarify and emphasis and more generally to regulate interactions with other people. From the psychology point of view, remarkable effort have been made to realize how humans interpret and perceive facial expressions in term of mental models of emotions and effective states. Over the past decade, we have witnessed extensive progress in machine learning and human computer interactions. Those progresses and the importance of Facial Expression Recognition in a number of applications in areas such as human computer interaction, multimedia communication, robotics and video surveillance, draw a lot of attention in computer vision community to utilise existing massive calculation tools and technologies to design algorithms in which humans interact with computers in a novel way. As this domain is still a challenging problem due to the complexity of movements of face components and temporal variations of expressions, in this research, our goal is to present a model that is able to fully model the temporal information as well as appearance features for recognizing six basic emotions (happy, sadness, anger, fear, surprise, disgust). Since lack of data is major problem in the area of computer vision and machine learning, we study methods to overcome this challenge. We show how the large data requirements for deep learning can be alleviated using synthetic data focusing on facial expression recognition problem. We investigate the use of synthetic facial expression data to circumvent the large data requirements and report on the effects of this approach for deep learning based expression recognition. For efficient and effective learning process we have implemented and performed extensive experiments on different well-known algorithms and models including, LBP TOP, Support vector machines, 3D neural networks and Recurrent neural networks. We evaluate and compare our methods on different publicly available datasets where the performances, merits and demerits are discussed and showed.

Date	15 Nov 2020
Original language	American English
Awarding Institution	École de technologie supérieure
Supervisor	Sylvie Ratté (Supervisor)