Passer à la navigation principale Passer à la recherche Passer au contenu principal

TCAM: Temporal Class Activation Maps for Object Localization in Weakly-Labeled Unconstrained Videos

  • École de technologie supérieure
  • McGill University

Résultats de recherche: Chapitre dans un livre, rapport, actes de conférence › Participation à un ouvrage collectif lié à un colloque ou une conférence › Revue par des pairs

14 Citations (Scopus)

Résumé

Weakly supervised video object localization (WSVOL) allows locating object in videos using only global video tags such as object classes. State-of-art methods rely on multiple independent stages, where initial spatio-temporal proposals are generated using visual and motion cues, and then prominent objects are identified and refined. The localization involves solving an optimization problem over one or more videos, and video tags are typically used for video clustering. This process requires a model per video or per class making for costly inference. Moreover, localized regions are not necessary discriminant because these methods rely on unsupervised motion methods like optical flow, or discarded video tags from optimization. In this paper, we leverage the successful class activation mapping (CAM) methods, designed for WSOL based on still images. A new Temporal CAM (TCAM) method is introduced for training a discriminant deep learning (DL) model to exploit spatio-temporal information in videos, using an CAM-Temporal Max Pooling (CAM-TMP) aggregation mechanism over consecutive CAMs. In particular, activations of regions of interest (ROIs) are collected from CAMs produced by a pretrained CNN classifier, and generate pixel-wise pseudo-labels for training a decoder. In addition, a global unsupervised size constraint, and local constraint such as CRF are used to yield more accurate CAMs. Inference over single independent frames allows parallel processing of a clip of frames, and real-time localization. Extensive experiments1 on two challenging YouTube-Objects datasets with unconstrained videos indicate that CAM methods (trained on independent frames) can yield decent localization accuracy. Our proposed TCAM method achieves a new state-of-art in WSVOL accuracy, and visual results suggest that it can be adapted for subsequent tasks, such as object detection and tracking.

Langue originaleAnglais
titreProceedings - 2023 IEEE Winter Conference on Applications of Computer Vision, WACV 2023
EditeurInstitute of Electrical and Electronics Engineers Inc.
Pages137-146
Nombre de pages10
ISBN (Electronique)9781665493468
Les DOIs
ÉtatPublié - 2023
Evénement23rd IEEE/CVF Winter Conference on Applications of Computer Vision, WACV 2023 - Waikoloa, Etats-Unis
Durée: 3 janv. 2023 → 7 janv. 2023

Série de publications

NomProceedings - 2023 IEEE Winter Conference on Applications of Computer Vision, WACV 2023

Conférence

Conférence23rd IEEE/CVF Winter Conference on Applications of Computer Vision, WACV 2023
Pays/TerritoireEtats-Unis
La villeWaikoloa
Période3/01/23 → 7/01/23

Classification disciplinaire ASJC Scopus

  • Intelligence artificielle
  • Applications de l'informatique
  • Vision par ordinateur et reconnaissance de formes

Empreinte digitale

Voici les principaux termes ou expressions associés à « TCAM: Temporal Class Activation Maps for Object Localization in Weakly-Labeled Unconstrained Videos ». Ces libellés thématiques sont générés à partir du titre et du résumé de la publication. Ensemble, ils forment une empreinte digitale unique.

Citer cette ressource