Skip to main navigation Skip to search Skip to main content

Navigation Agents for the Visually Impaired: A Sidewalk Simulator and Experiments

  • Martin Weiss
  • , Simon Chamorro
  • , Roger Girgis
  • , Margaux Luck
  • , Samira E. Kahou
  • , Joseph P. Cohen
  • , Derek Nowrouzezahrai
  • , Doina Precup
  • , Florian Golemo
  • , Chris Pal
  • University of Montreal
  • Université de Sherbrooke
  • Polytechnique Montréal
  • McGill University
  • ElementAI

Research output: Contribution to journal › Conference article › peer-review

9 Citations (Scopus)

Abstract

Millions of blind and visually-impaired (BVI) people navigate urban environments everyday, using smartphones for high-level path-planning and white canes or guide dogs for local information. However, many BVI people still struggle to travel to new places. In our endeavour to create a navigation assistant for the BVI, we found that existing Reinforcement Learning (RL) environments were unsuitable for the task. This work introduces SEVN, a sidewalk simulation environment and a neural network-based approach to creating a navigation agent. SEVN contains panoramic images with labels for house numbers, doors, and street name signs, and formulations for several navigation tasks. We study the performance of an RL algorithm (PPO) in this setting. Our policy model fuses multi-modal observations in the form of variable resolution images, visible text, and simulated GPS data to navigate to a goal door. We hope that this dataset, simulator, and experimental results will provide a foundation for further research into the creation of agents that can assist members of the BVI community with outdoor navigation.

Original languageEnglish
Pages (from-to)1314-1327
Number of pages14
JournalProceedings of Machine Learning Research
Volume100
Publication statusPublished - 2019
Externally publishedYes
Event3rd Conference on Robot Learning, CoRL 2019 - Osaka, Japan
Duration: 30 Oct 2019 → 1 Nov 2019

Keywords

  • computer vision
  • dataset
  • outdoor navigation
  • reinforcement learning

ASJC Scopus subject areas

  • Software
  • Control and Systems Engineering
  • Statistics and Probability
  • Artificial Intelligence

Fingerprint

Dive into the research topics of 'Navigation Agents for the Visually Impaired: A Sidewalk Simulator and Experiments'. These topics are generated from the title and abstract of the publication. Together, they form a unique fingerprint.

Cite this