Passer à la navigation principale Passer à la recherche Passer au contenu principal

MoP-CLIP: A Mixture of Prompt-Tuned CLIP Models for Domain Incremental Learning

  • École de technologie supérieure
  • Thales Digital Solutions

Résultats de recherche: Chapitre dans un livre, rapport, actes de conférence › Participation à un ouvrage collectif lié à un colloque ou une conférence › Revue par des pairs

8 Citations (Scopus)

Résumé

Despite the recent progress in incremental learning, addressing catastrophic forgetting under distributional drift is still an open and important problem. Indeed, while state-of-the-art domain incremental learning (DIL) methods perform satisfactorily within known domains, their performance largely degrades in the presence of novel domains. This limitation hampers their generalizability, and restricts their scalability to more realistic settings where train and test data are drawn from different distributions. To address these limitations, we present a novel DIL approach based on a mixture of prompt-tuned CLIP models (MoPCLIP), which generalizes the paradigm of S-Prompting to handle both in-distribution and out-of-distribution data at inference. In particular, at the training stage we model the features distribution of every class in each domain, learning individual text and visual prompts to adapt to a given domain. At inference, the learned distributions allow us to identify whether a given test sample belongs to a known domain, selecting the correct prompt for the classification task, or from an unseen domain, leveraging a mixture of the prompt-tuned CLIP models. Our empirical evaluation reveals the poor performance of existing DIL methods under domain shift, and suggests that the proposed MoP-CLIP performs competitively in the standard DIL settings while outperforming state-of-the-art methods in OOD scenarios. These results demonstrate the superiority of MoP-CLIP, offering a robust and general solution to the problem of domain incremental learning.

Langue originaleAnglais
titreProceedings - 2024 IEEE Winter Conference on Applications of Computer Vision, WACV 2024
EditeurInstitute of Electrical and Electronics Engineers Inc.
Pages1751-1761
Nombre de pages11
ISBN (Electronique)9798350318920
Les DOIs
ÉtatPublié - 3 janv. 2024
Evénement2024 IEEE Winter Conference on Applications of Computer Vision, WACV 2024 - Waikoloa, Etats-Unis
Durée: 4 janv. 2024 → 8 janv. 2024

Série de publications

NomProceedings - 2024 IEEE Winter Conference on Applications of Computer Vision, WACV 2024

Conférence

Conférence2024 IEEE Winter Conference on Applications of Computer Vision, WACV 2024
Pays/TerritoireEtats-Unis
La villeWaikoloa
Période4/01/24 → 8/01/24

Classification disciplinaire ASJC Scopus

  • Intelligence artificielle
  • Applications de l'informatique
  • Vision par ordinateur et reconnaissance de formes

Empreinte digitale

Voici les principaux termes ou expressions associés à « MoP-CLIP: A Mixture of Prompt-Tuned CLIP Models for Domain Incremental Learning ». Ces libellés thématiques sont générés à partir du titre et du résumé de la publication. Ensemble, ils forment une empreinte digitale unique.

Citer cette ressource