Learning to localize objects with limited supervision

Abstract

Deep object detectors are omnipresent in today’s industries meeting many requirements like inspection, automation, surveillance, monitoring, etc. One of the important bottlenecks in developing today’s object detection systems is the need for a huge collection of labeled data to train the data-hungry deep detectors. Labeling images with bounding boxes is expensive and time-consuming. The cost is prohibitively high when strong expertise is needed for labeling, for example, a radiologist labeling medical images. Also, the current successful fully supervised detectors won’t scale for our growing demand as it is practically impossible to label thousands of images for every object detection task we want to solve. Thus to mitigate this annotation challenge for training modern deep detectors, the community is exploring many directions including weakly supervised, self-supervised, semi-supervised, domain adaptation, and fewshot training. To contribute to this effort, in this thesis, we explored weakly supervised and semi-supervised methods for training localization systems for single-object and multi-object localization. We identified the limitations and the training difficulties of the current main-stream weakly supervised and semi-supervised detection techniques. Then we proposed alternate designs and training techniques to mitigate this. The first contribution of this thesis is a fully convolutional weakly supervised object localization architecture with a learnable localization component. Different from the default architecture CAM(ClassActivation Mapping), our method is fully convolutional and has separate components for learning localization. We used spatial transformers in a convolutional fashion for learning the localization with affine transformations. One of the major challenges of weakly supervised localization methods is the localization focus on discriminative regions. In our architecture, this can be reduced easily by a regularization constraint on the learned parameters. With extensive empirical studies, we established improved localization and flexible control of our proposed fully convolutional weakly supervised localization network. The second contribution is a method to reuse the existing fully supervised object detection architectures for weak supervision. While there are plenty of object detection architectures proposed for supervised object detection, researchers working on weakly supervised detectors use the WSDDN (Weakly Supervised Deep Detection Networks) architecture because it is straightforward to train WSDDN with weak image-level labels. WSDDN computes class probabilities of region proposals and aggregates these probabilities to produce image-level class probabilities. Given only the global image-level labels in weakly supervised settings, there is no efficient technique to label object regions using them so as to train the system using fully supervised detectors. To address this, we proposed a sampling-based pseudo-label construction method, using which region-level labels are derived from the image labels. With these pseudo-labels for regions, we can train any off-the-shelf fully supervised detection method, thus eliminating the need for customized architectures for weakly supervised object detection. We also showed that the performance of the sampling-based detector can be improved significantly by using a few annotated images. The third contribution is about adapting the main-stream semi-supervised detectors to train on high-resolution aerial images. Aerial image object detection can benefit significantly if effective semi-supervised detectors can be designed because plenty of images are collected in surveillance applications using drones and satellites. Those images are simply not used during training because of the lack of annotations. We observed that the tiny objects in high-resolution aerial images cannot be pseudo-labeled effectively for semi-supervised training. To fix this, our proposed zoom-in detector uses density crops where regions with high concentrations of small objects are identified and cropped out. These regions are then upscaled for better detection of small objects. This density crop-based training is implemented within the detector giving more pseudo labels for tiny objects which translates to improved semi-supervised object detection on aerial images.

Date	10 Nov 2023
Original language	American English
Awarding Institution	École de technologie supérieure
Supervisor	Marco Pedersoli (Supervisor) & Éric Granger (Co-supervisor)