Towards local classifier generation for dynamic selection

Abstract

Multiple Classifier Systems (MCS) focus on the combination of classifiers to achieve better performance than a single robust one. These systems unravel three major phases: pool of classifiers generation, Static or Dynamic selection and integration. Dynamic Selection (DS) is an active research topic in the field of MCS. It’s based on the selection of the most competent classifier(s), on the fly, for every test sample. To operate, the DS scheme needs to be provided with a pool of classifiers, it will then compute the region of competence defined as a set of the test sample’s nearest neighbors (in most of DS techniques) that determines the competence of the classifiers. The regions of competence located on the decision boundaries are called "indecision regions" and usually contribute into the struggle of DS technique into always selecting the most competent classifiers. On the other hand, Dynamic Selection relies on classifier generation techniques that are meant for static combinations. In other words, these methods adopt a global approach into generating the classifiers, which contradicts the local aspect that defines the dynamic selection scheme. Therefore, in this work we address the problem of covering the indecision regions with locally competent classifiers in the context of dynamic selection. We propose a system that exploits local information to build classifiers that cross the indecision regions of competence guaranteeing a separation between the samples from different classes (frienemies). We also proposed five novel local selection strategies to construct pools of these classifiers, that is adapted to the Dynamic selection system. In generalization, we rely on the recommendations of our previous work, where an investigation was conducted on the reasons behind the out-performance of the DS techniques over the K-NN classifier even though, most of the techniques rely on the definition of the nearest neighbors as a competence region. It was concluded that DS techniques deal better with instances located in indecision regions according to an instance hardness measure whereas the K-NN is well suited for the samples with a low degree of instance hardness. Therefore, we exploited the concept of the hardness of samples to determine whether to use the K-NN of DS techniques for classification. Experiments were conducted using several DS techniques under the five proposed pool generation approaches. The results of this research have shown that focusing on a local approach to generate classifiers for Dynamic Selection is a promising path into guaranteeing the existence of locally competent classifiers as it outperformed the results of the literature for certain techniques. When there is no improvement, the results remain comparable to the literature’s. Yet, there is still room for improvement for such approaches and further investigations will be lead towards the local pool generation for Dynamic Selection.

Date	3 Dec 2018
Original language	American English
Awarding Institution	École de technologie supérieure
Supervisor	Robert Sabourin (Supervisor) & George D.C. Cavalcanti (Co-supervisor)