Domain adaptation with missing data for medical image segmentation

Abstract

Domain Adaption (DA) methods have recently attracted substantial attention in computer vision as they improve the transferability of deep network models from a source to a target domain with different characteristics. DA is key in mitigating the need for laborious pixel annotations required by deep segmentation models. However, the framework of most common DA methods is not realistic, as it requires access to whole datasets, both in the source and in the target domain. Yet in clinical settings, only a few or even a single target sample(s) are typically available, while the source data might be inaccessible. Therefore, this thesis main objective is to propose domain adaptation algorithms to segment medical images with limited training datasets. In our first objective, we explore adapting segmentation networks with inequality constraints on the network predictions of target samples. Thereby, we implicitly match the prediction statistics of the target and source domains, with permitted uncertainty of prior knowledge. We address the ensuing constrained optimization problem with differentiable penalties, fully suited for conventional stochastic gradient descent approaches. In our second objective, we introduce source-free domain adaptation for image segmentation. Our formulation is based on minimizing a label-free entropy loss defined over target-domain data, which we further guide with a domain-invariant class-ratio prior on the segmentation regions. The prior is estimated from anatomical knowledge and integrated in the form of a Kullback–Leibler divergence in our overall loss function. We show the effectiveness of our prior-aware entropy minimization in various source-free domain adaptation scenarios, with different modalities and applications, including spine, prostate and cardiac segmentation. In our third objective, we study a source-free adaptation method for use at test-time with a single target subject. We investigate shape-guided entropy minimization objectives. We explore the potential of integrating various constraints in the form of shape moments, to guide domain adaptation towards plausible solutions. In particular, we exploit the size, centroid, and distance-to-centroid of anatomical structures through penalty constraints in our overall loss function. In our applications, an estimation of these shape moments is derived from textbook anatomical knowledge. Our method is validated in two challenging source-free single-subject adaptation tasks: MRI-to-CT adaptation for cardiac images, and cross-site adaptation for prostate images. The efficiency of 2D and 3D shape constraints are demonstrated in both applications. In conclusion, each objective progressively pushes further the complexity of the adaptation task and the amount of missing data, to achieve a realistic clinical setting. The proposed methods address this challenge by studying how to best leverage domain knowledge such as anatomical shape knowledge. This thesis led to six different publications as first author, including three at MICCAI conferences, two journal publications, one in IEEE Transactions for Medical Imaging and one in Medical Image Analysis (MedIA), and one ongoing submission to MedIA. All the codes ensuing from this thesis are publicly available, and free to reuse and modify. The functional programming style used makes it easy to integrate new loss functions and shape information, with little-to-no additional coding efforts.

Date	23 Jun 2023
Original language	American English
Awarding Institution	École de technologie supérieure
Supervisor	Ismail Ben Ayed (Supervisor) & Hervé Lombaert (Co-supervisor)