Multi-target domain adaptation for person re-identification

Abstract

Re-identification of individuals (ReID) involves matching images of pedestrians captured by a network of multiple cameras that do not capture conditions or fields of view. Given a query image captured for an individual, the system must automatically find all other images of the same individual in a pedestrian image gallery. Several factors make this task difficult, such as the difference in camera viewpoints, visual occlusions, pose, and illumination variations over time. This research area has important applications in the automation of video surveillance monitoring and biometrics. Recent advances in deep learning and the availability of large annotated databases have enabled modern systems to train efficiently and achieve very high accuracy. Although these solutions work well with sufficient training data captured for a given camera network, also known as a "domain", performance drops significantly when processing images from another operational domain. In the application of a ReID system, we consider two types of data: source data coming from a controlled environment that is labeled, and target data, generally unlabeled, coming from the operational domain in which the system is deployed. Indeed, the different domains diverge because of their different characteristics, such as camera position, image resolution, lighting, and background, to name a few. Since the cost of generating a new labeled dataset for each new operational domain is very high, unsupervised domain adaptation (UDA) techniques have been developed to leverage data from a labeled source dataset and an unlabeled target dataset to improve performance on the operational data. However, another problem arises when multiple target domains are present. Each target domain diverges in its specific characteristics and requires a unique adaptation. Adapting a customized ReID model for each target domain is a simple but impractical solution in real applications where computing resources are limited. On the other hand, training a single model on all targets simultaneously can reduce accuracy due to insufficient model capacity, when dealing with highly varied data. The optimal solution must therefore have high accuracy on each target while minimizing the memory complexity of the resulting model. Very few works tackle multi-target unsupervised domain adaptation (MTDA) methods for person ReID. These methods are often not accurate and do not consider the complexity of the solution during evaluation. The success of a system depends on the data used, the capacity of the CNN, and the MTDA method employed. Therefore, it is essential to consider the complexity of the trained CNN as well as the datasets used when evaluating an MTDA method. This work tackles the MTDA problem for person ReID using Knowledge Distillation (KD). The objective is to train a compact CNN model capable of matching two pedestrian images captured by different cameras. The Deep Learning (DL) model will be able to perform this task on images from several target domains. We evaluate the solution according to (1) the accuracy of the model when trained for many target domains, and (2) the time and memory complexity during inference. In this dissertation, we present two main contributions related to these criteria. The first contribution, presented in Chapter 4, is the development of a knowledge distillationbased MTDA technique for person ReID entitled KD-ReID. Using a cost function tailored specifically for ReID, we adapt a set of "Teacher" CNN models, each one to a specific target domain, and then distill the knowledge to a single "Student" CNN model. The resulting model is accurate for all target domains while remaining low cost for a real-world application. We show that this approach outperforms existing state-of-the-art approaches in terms of accuracy and model complexity. Furthermore, we show that KD-ReID is very flexible, allowing us to use Teacher models of different architecture and training techniques. This flexibility reinforces the potential of KD-ReID to be used in real applications. This contribution has been accepted and published in the IEEE International Conference on Image Processing 2022 (ICIP2022). The second contribution, presented in Chapter 5, is a comprehensive comparative study of KD-MTDA techniques, to maximize the relationship between system complexity and accuracy for the ReID task. Using a compact model, we adapt various techniques to our MTDA problem. More precisely, we analyze and compare four techniques: domain-specific BN layers, multibranch model, domain adapters, and distillation adapters. The techniques studied aim to increase accuracy on multiple targets at the cost of increasing the number of model parameters. In addition to allowing optimization of our solution in a resource-constrained situation, this study allows us to draw important conclusions on the trade-off between the accuracy of specialized MTDA methods for our task.

Date	8 Aug 2023
Original language	American English
Awarding Institution	École de technologie supérieure
Supervisor	Éric Granger (Supervisor) & Rafael Menelau Oliveira Cruz (Co-supervisor)