Segmentation is a critical task in medical image analysis, which plays a vital role in computeraided diagnose, image-guided radiotherapy, and surgical navigation. Deep learning-based segmentation methods have achieved unprecedented progress in recent years, benefiting from large amounts of annotated data. However, obtaining annotations for medical images requires substantial efforts and costs. Further, the limited availability of annotated medical data poses a significant challenge in achieving high-performance medical image segmentation. To obtain competitive performance with limited labeled data, semi-supervised learning approaches have been developed to also exploit unlabeled data. Though these approaches have achieved an improved performance, the unsupervised training on unlabeled data also brought some challenges. For instance, inaccurate predictions made for unlabeled data in the initial training stage can be accentuated as the training progresses, leading to a degradation in performance. Another problem is the labeled data may be insufficient for the model to learn an anatomical-plausible shape for the organ to segment. In such case, it may be useful to employ anatomical priors to guide the model learning. Another practical limitation is that the labeled data and unlabeled data can have distinct distributions due to differences in the image acquisition devices. This represents a more challenging task since the model’s training can be dominated by labeled data and, as a result consequently, this model may fail to adapt to the distribution of target data. Several approaches have been proposed for domain adaptation, for instance, relying on auxiliary reconstruction decoders or style-transfer. However, developing simple yet highly effective solutions for this task, that avoid the use of a complex framework, is still a pressing direction of research.
The main objective of this thesis is to develop simple and accurate methods for medical image segmentation under these two challenging scenarios. Specifically, we first proposed a self-paced and self-consistent co-training method for semi-supervised image segmentation. This method addresses the problem of inaccurate predictions for unlabeled data during the initial training stage, thereby boosting segmentation performance. Secondly, we developed a constrained adversarial training method for semi-supervised segmentation, which enforces anatomical-plausible predictions by incorporating complex non-differentiable anatomical priors. The last contribution focuses on the more challenging domain adaptation scenario. For this task, we proposed a shape-aware joint distribution alignment method for cross-domain image segmentation, which achieves competitive cross-domain segmentation performance by explicitly aligning domain-invariant representation encoding shape size and spatial relationship between classes. This thesis has resulted in three publications in high-impact medical imaging journals as well as a publication in a top conference of that field. The specific objectives of this thesis are presented below.
In our first objective, we focus on semi-supervised segmentation and propose a method based on a co-training framework. First, we present a self-paced learning strategy for co-training that enables jointly-trained neural networks to focus on easier-to-segment regions first, and then gradually consider harder ones. This strategy is implemented via an end-to-end differentiable loss in the form of a generalized Jensen Shannon Divergence (JSD). To encourage the networks to produce not only consistent but also confident predictions, we enhance this generalized JSD loss with an uncertainty regularizer based on entropy. Furthermore, the robustness of individual models in our co-training framework is further improved using a self-ensembling loss that enforces the models prediction to be consistent across different training iterations. The effectiveness of our method is assessed on three challenging segmentation datasets including images of different modalities, for which it boosts segmentation accuracy when very few labeled images are used. We also explore the impact of the proposed self-paced learning strategy, self-consistency strategy, as well as our uncertainty regularizer. Experimental results show the effectiveness of each component in the proposed method.
Our second objective also focuses on semi-supervised segmentation. For this objective, a constrained adversarial training method is proposed for anatomical-plausible segmentation. Unlike approaches focusing solely on accuracy measures like Dice, this method considers complex anatomical constraints like connectivity, convexity, and symmetry that cannot be easily modeled in a loss function. The problem of non-differentiable constraints is solved using the Reinforce algorithm which enables to obtain a gradient for the violated constraints. To generate constraint-violating examples on the fly, and thus obtain useful gradients, our method adopts an adversarial training strategy which modifies training images to maximize the constraint loss, and then updates the network to be robust to these adversarial examples. The proposed method offers a generic and efficient way to add complex segmentation constraints on top of any segmentation network. Experiments on four clinically-relevant datasets as well as on synthetic datasets generated for this work demonstrate the effectiveness of our method in terms of segmentation accuracy and anatomical plausibility.
The last objective focuses on the domain adaptation scenario. For this scenario, we developed a shape-aware joint distribution alignment method for cross-domain segmentation, which aligns high-order statistics, computed for the source and target domains, that encode domaininvariant spatial relationships between segmentation classes. Our method first estimates the joint distribution of predictions for pairs of pixels whose relative position corresponds to a given spatial displacement. Domain adaptation is then achieved by aligning the joint distributions of source and target images, computed for a set of displacements. Two enhancements of this method are proposed. The first one uses an efficient multi-scale strategy that enables capturing long-range relationships in the statistics. The second one extends the joint distribution alignment loss to features in intermediate layers of the network by computing their cross-correlation. We test our method on the task of unpaired multi-modal cardiac segmentation using the MultiModality Whole Heart Segmentation (MMWHS) Challenge dataset and on the task of prostate segmentation task, where images of two datasets are taken as data from different domains. Our results show the advantages of our method compared to recent approaches for cross-domain image segmentation.
| Date | 28 Sept 2023 |
|---|
| Original language | American English |
|---|
| Awarding Institution | - École de technologie supérieure
|
|---|
| Supervisor | Christian Desrosiers (Supervisor), Marco Pedersoli (Co-supervisor) & Caiming Zhang (Co-supervisor) |
|---|