On Domain-Incremental Learning methods and its applications to forgery detection

Abstract

The easy access to powerful devices and quick spread of social networks have led to an unprecedented increase of the amount of available digital images. This has facilitated the rise of digital image forgery, which can be leveraged easily by criminals with obscure purposes (i.e., insurance fraud, identity theft, etc). To identify the most prevalent image forgery techniques, convolutional neural networks (CNN) have been proposed recently in the literature. Nevertheless, these approaches make strong assumptions about the availability of data and its domains. In particular, they assume that i) training and testing data are drawn from the same domain distribution, and ii) the data domain remains unchanged over time. We argue that these assumptions, however, may limit the applicability of existing forgery detection methods to highly constrained scenarios. To address these limitations, we present a novel Domain-Incremental Learning (DIL) approach based on a mixture of prompt-tuned CLIP models (MoP-CLIP), which generalizes the paradigm of S-Prompting to handle both in-distribution and out-of-distribution (OOD) data at inference. At the training stage, we model the feature distribution of every class in each domain, learning individual text and visual prompts to adapt to a given domain. At inference, the learned distributions allow us to identify whether a given test sample belongs to a known domain, selecting the correct prompt for the classification task, or from an unseen domain, leveraging a mixture of the prompt-tuned CLIP models. Our empirical evaluation reveals the limitations of existing DIL methods under domain shift, and suggests that the proposed MoP-CLIP performs competitively in the standard DIL settings while outperforming state-of-the-art methods in OOD scenarios. These results demonstrate the superiority of MoP-CLIP , offering a robust and general solution to the problem of domain incremental learning, while relaxing the assumptions previously made for data distributions. We also emphasize that domain-incremental learning approaches must be benchmarked with challenging real-world datasets, and therefore conduct a realistic evaluation of the proposed method, as well as existing domain-incremental approaches, on a harder task, i.e., domainincremental forgery detection, Our findings reveal that in this challenging scenario, the proposed method still yields competitive performance.

Date	14 Sept 2023
Original language	American English
Awarding Institution	École de technologie supérieure
Supervisor	José Dolz (Supervisor) & Christian Desrosiers (Co-supervisor)