Learning visual recognition models with limited data

Abstract

Deep learning, particularly through deep neural networks, has achieved remarkable success in computer vision. Large-scale models with millions of parameters have revolutionized the field, capturing complex patterns and improving performance across tasks like image classification, object detection, or semantic segmentation. Training with extensive datasets is key to enhancing model generalization, enabling accurate predictions on new data and adaptability to real-world complexities. However, despite the exceptional benefits, challenges arise due to the cost associated with data acquisition and the potential distribution shift between train and test data. This thesis aims at tackling those challenges and explores different ways to optimize the learning and adaptation of deep neural networks while maintaining or enhancing performance. In a first work, we explore the usage of generative models to generate images useful for a downstream task. More particularly, we leverage the power of generative adversarial networks (GAN) to generate new augmented samples useful to improve the training of an image classifier and increase its robustness and performance. Unlike traditional heuristic transformations, the approach presented learns data augmentation directly from training data using an encoder-decoder architecture and a spatial transformer network, producing more complex samples within the same class. In a second work, we further explore data augmentation and propose an efficient approach to reduce the computational power needed to define the best data augmentation parameters, improving generalization without requiring domain knowledge or an exhaustive search. We optimize augmentation parameters using a validation set through bi-level optimization, removing the need for an expensive external validation loop. We validated the method on natural images but also on histological images. Finally, in a third work, we explore test-time adaptation (TTA) and present a categorization of selected orthogonal TTA techniques interesting for adapting models to data drifts, such as small batch normalization, stream rebalancing, reliable sample selection, and network confidence calibration. We give insights into their impact on different scenarios, highlighting trade-offs in accuracy, computational power, and model complexity, while also revealing the synergies that arise from combining techniques. The presented works open up new avenues for further research, offering insights and practical solutions for training and adapting deep neural networks under challenging conditions.

Date	13 Oct 2023
Original language	American English
Awarding Institution	École de technologie supérieure
Supervisor	Marco Pedersoli (Supervisor) & Ismail Ben Ayed (Co-supervisor)