Évaluation réaliste de l’apprentissage Few-Shot transductif

Abstract

The Few-Shot learning community has a strong research interest in transductive methods. Transductive inferance exploits both labeled data from the support set and unlabeled data from the query set. These methods were able to significantly surpass the performance of inductive methods. We question the performances of these methods based on the fact that the current evaluation process might not be adequate. The methods are evaluated based on their ability to generalize in order to correctly classify the query set’s data. In the current literature, the query set’s class distribution is uniform. Knowing beforehand the class distribution of the data to be classified does not reflect the challenge of a realistic classification problem. Moreover, the inference of certain transductive methods is carried out by exploiting this false a priori information. In order to produce random classification tasks, we propose to sample the marginal probabilities of the query set’s classes by following the Dirichlet’s distribution. We evaluated state-of-the-art methods on three benchmarks : mini-Imagenet, tiered-Imagenet, and Caltech-UCSD Birds 200 (CUB) and using two neural network architectures : ResNet-18 and WRN28-10. All transductive methods suffer from performance loss when evaluated on randomly distributed tasks. It’s also interesting to observe that the best performing methods in the current FSL literature are the ones that suffer the most from this test scenario, undergoing performance losses of up to almost 20%. We propose a transductive method (a-TIM) to tackle this new realistic classification problem. We use the a-divergence to generalize the loss function exploiting mutual information. This divergence’s shape is appropriate to deal with different severities of class imbalance. Moreover, our experimental results demonstrate that our method outperforms all corresponding state-of-the-art methods. The code of this project can be found in the following GitHub repository : https://github.com/oveilleux/Realistic_Transductive_Few_Shot.

Date	10 May 2022
Original language	French
Awarding Institution	École de technologie supérieure
Supervisor	Ismail Ben Ayed (Supervisor)