Recherche de l’information dans les réseaux de neurones convolutifs pré-entraînés

Abstract

This thesis evaluates the use of pre-trained convolutional neural networks (CNNs) as generic feature extractors for new image classification contexts, a strategy known as transfer learning. A number of open research questions are addressed, given the increasing availability of diverse, high performance, pre-trained CNN architectures for transfer learning : Which networks and network activation layers are most effective for transfer learning ? Can different networks be combined to improve classification ? How does the effectiveness of classification differ across data contexts, i.e. broad visual categories (e.g. buildings, cars) vs. specific imaging contexts (e.g. face or brain scans of the same person or family members)? A generic memory-based classification model is proposed in order to evaluate and compare the accuracy of CNN architectures, where generic activation maps from arbitrary networks serve as image features and classification is achieved via nearest neighbor indexing. A number of feature pooling and normalization schemes are evaluated, including maximum, average and generalised mean pooling, and normalisation schemes include raw activation maps, L2 normalization. Finally, a binary feature encoding scheme is proposed to compress data and improve classification accuracy, where individual activation features are binarized in order to maximize information gain. As an additional baseline, classification is also evaluated using traditional hand crafted image features extracted via the scale-invariant feature transform (SIFT). Evaluation compares a large, comprehensive list of existing CNN architectures, all pre-trained on the standard ImageNet dataset (1000 categories x 1000 images), including VGG, Inception, ResNet, Xception, DenseNet, MobileNet, NasLarge, NasMobile etc. In order to avoid potential bias towards data used in CNN pre-training, classification experiments are based upon independent image datasets and categories unrelated to ImageNet data. These include broad visual categories from Caltech101 dataset (Fei-Fei et al. (2004)), and of specific contexts including human face images from the FERET dataset (Phillips et al. (1998)) and brain magnetic resonance images of the human brain from the Human Connectome Project (HCP) (Van Essen et al. (2013)). For general categories (Caltech101 data), the highest accuracies for each network range from (73.29% to 93.02%) for networks (NasMobile layer 739) and (DenseNet201 layer 704), respectively, consistent with state-of-the-art performance. Concatenating high-accuracy layers from different networks generally increases accuracy, the highest accuracy (94.01%) was achieved combining (Xnception-Resnet-DenseNet-InceptionResNetV2). For specific face instances (FERET data), the highest recognition accuracy (perfect 100%) is achieved from filter responses within network layers and for 2D SIFT matching. Filter responses at the network output are less accurate (98%). Binarization improves the accuracy of gender classification (InceptionV3 with binarization, AUC=0.981) vs. (InceptionV3 without binarization, AUC=0.966), improving upon 2D SIFT (AUC=0.926). For human brain instances (HCP data), pre-trained CNN features combined with 3D SIFT features achieves state-of-theart accuracy for binary gender classification (DenseNet201 with binarization, AUC=0.987), and family member classification for 1010 subjects and 400 families is (DenseNet201 with binarization, AUC=0.925).

Date	2 Oct 2020
Original language	French
Awarding Institution	École de technologie supérieure
Supervisor	Matthew Toews (Supervisor)