Several studies offer unsupervised learning models that allow one to discover a variety of generic dataset representations. However, optimizing the objective function of these models does not ensure that disentangled representations are obtained that are explicitly useful on subsequent related tasks. In order to effectively compare different representations obtained, a method for quantitatively measuring disentanglement is needed. Various metrics addressing this problem have been proposed. However, it is observed that they are often inconsistent when compared to each other or compared to a practitioner's subjective assessment. Comparing metrics is difficult in a typical representation learning context, since the generic nature of the representations obtained prevents knowing with certainty the real quality of the measured properties. In order to make metrics reliable, it is important to demystify these inconsistencies.
In order to remedy this problem, this work proposes to characterize the metrics on representations whose representative properties are known. First, a metric taxonomy is put in place to help identify the similarities between them. This taxonomy, however, is not sufficient to understand the disagreement between metrics. Desirable metric properties are defined, and the proposed scenarios are used to characterize against these properties.
In this document, it is discovered that several metrics have difficulty in correctly measuring properties of which they should be able to provide measurements. We identify DCI as the most robust in identifying disentangling according to the explicit quality, modularity and compactness of a representation. In our representative experimental scenarios, DCI avoids several instabilities of various causes. DCI can therefore be used without fear of its compatibility with the data set and the representation in which the practitioner wishes to measure disentanglement therein. Finally, we further discuss key differences between experimental and actual data sets as well as various practical considerations & identify further possibilities for improvement in future studies.
| Date | 30 May 2022 |
|---|
| Original language | French |
|---|
| Awarding Institution | - École de technologie supérieure
|
|---|
| Supervisor | Ghyslain Gagnon (Supervisor) |
|---|
Boilard, J. (Author),
Gagnon (Supervisor),
30 May 2022Student thesis: Master's thesis › Master in Engineering: Electrical Engineering