Skip to main navigation Skip to search Skip to main content

Adversarial Robustness Via Fisher-Rao Regularization

  • Marine Picot
  • , Francisco Messina
  • , Malik Boudiaf
  • , Fabrice Labeau
  • , Ismail Ben Ayed
  • , Pablo Piantanida
  • McGill University
  • Laboratoire des Signaux et Systèmes
  • École de technologie supérieure

Research output: Contribution to journal › Journal Article › peer-review

19 Citations (Scopus)

Abstract

Adversarial robustness has become a topic of growing interest in machine learning since it was observed that neural networks tend to be brittle. We propose an information-geometric formulation of adversarial defense and introduce Fire, a new Fisher-Rao regularization for the categorical cross-entropy loss, which is based on the geodesic distance between the softmax outputs corresponding to natural and perturbed input features. Based on the information-geometric properties of the class of softmax distributions, we derive an explicit characterization of the Fisher-Rao Distance (FRD) for the binary and multiclass cases, and draw some interesting properties as well as connections with standard regularization metrics. Furthermore, we verify on a simple linear and Gaussian model, that all Pareto-optimal points in the accuracy-robustness region can be reached by Fire while other state-of-the-art methods fail. Empirically, we evaluate the performance of various classifiers trained with the proposed loss on standard datasets, showing up to a simultaneous 1% of improvement in terms of clean and robust performances while reducing the training time by 20% over the best-performing methods.

Original languageEnglish
Pages (from-to)2698-2710
Number of pages13
JournalIEEE Transactions on Pattern Analysis and Machine Intelligence
Volume45
Issue number3
DOIs
Publication statusPublished - 1 Mar 2023

Keywords

  • Adversarial regularization
  • adversarial training
  • computer vision
  • deep learning
  • fisher-rao distance
  • information geometry
  • neural networks
  • safety AI

ASJC Scopus subject areas

  • Software
  • Computer Vision and Pattern Recognition
  • Computational Theory and Mathematics
  • Applied Mathematics
  • Artificial Intelligence

Fingerprint

Dive into the research topics of 'Adversarial Robustness Via Fisher-Rao Regularization'. These topics are generated from the title and abstract of the publication. Together, they form a unique fingerprint.

Cite this