Modèle basé sur le Transformer pour une détection robuste du visage des nourrissons et des enfants hospitalisés en utilisant des images RVB et thermiques

Abstract

Face localization in videos of patients in the Pediatric Intensive Care Unit (PICU) is an essential step in several applications of video-based non-invasive patient monitoring. These applications range from assessing the patient’s pain from facial expression to estimating the heart and respiratory rate from facial features. The localization accuracy of the patients’ faces can impact the quality of the patient monitoring application. Currently, Convolutional Neural Network (CNN) based face detection models, such as RetinaFace, achieve high accuracy in general settings. However, their accuracy substantially declines when applied in the PICU or in the Neonatal Intensive Care Unit (NICU). Such decline can be attributed to the challenging clinical setting. Particularly, occluded patient face, variable lighting conditions, and extreme patient pose. Addressing this, we use a transformer-based detection model DEtection TRansformer (Detr) pre-trained on the WiderFace dataset to detect faces in the PICU. Our results show that the Detr model compared to RetinaFace generalizes very well to the PICU data. Moreover, we unveiled a novel approach integrating weakly aligned RGB and thermal images, boosting detection accuracy for both Detr and Retinaface. Leveraging both thermal and RGB images, a pre-trained Detr outperformed RetinaFace by 15.3% reaching an Average Precision (AP) of 71.6%. Finally, we discuss the results of fine-tuning both models on 282 images of diverse patients of different ages and poses in the PICU. The transformer-based model Detr generalizes better than the CNN-based RetinaFace model in detecting the faces in the PICU.

Date	2 May 2024
Original language	French
Awarding Institution	École de technologie supérieure
Supervisor	Rita Noumeir (Supervisor) & Philippe Jovet (Co-supervisor)