Détection de comportements et d’événements potentiellement mortels dans les prisons à partir d’analyse de vidéos par intelligence artificielle

Abstract

Suicide is a complex phenomenon. According to the World Health Organization (WHO), a person kills itself every 40 seconds, for a total of 800,000 deaths per year. This is even more frequent in prison. The most common self-harm methods are hanging then wrist-cutting. If detected soon enough, suicide attempts can lead to an early hospitalization and a likely survival. Closed-circuit television is a potential solution, but, a security agent loses 95% of its attention in a twenty-minute span when working with multiple cameras. Human surveillance also raises ethical preoccupations. This works aims to automate surveillance by detecting dangerous behaviors. No facial or identity recognition is performed. Data are not recorded. We propose a human action recognition framework. Red-green-blue + depth (RGB+D) cameras, such as the Microsoft Kinect, are powerful new tools for action recognition. They propose different streams : RGB video, infrared, three-dimensional (3D) point clouds, and estimated human skeleton. RGB images are not suited to security applications as they do not yield any information in the dark. Skeleton data are an interesting representation because of their low dimensionality and their insensibility to background information, but prove insufficient when detecting object-related actions or actions with similar motions. Suicides fall into this category. We find infrared videos to be a strong alternative while working in the dark. A deep learning architecture is proposed, combining infrared video with 3D pose data. Two independent modules extract features from each stream. A third and final module fuses those features and studies them conjointly before emitting a final prediction. We achieve state-of-the-art results on the largest RGB+D action recognition dataset to date : NTU RGB+D. Those results are then transferred from an offline context to an online early prediction network. The new network can then be used for online action detection, which mimics real-time scenarios, with an additional temporal segment proposal module. Again, we achieve state-of-the-art results on two benchmark datasets : NTU RGB+D and PKU-MMD.

Date	16 Jun 2020
Original language	French
Awarding Institution	École de technologie supérieure
Supervisor	Rita Noumeir (Supervisor)