Skip to main navigation Skip to search Skip to main content

Improving video captioning for deaf and hearing-impaired people based on eye movement and attention overload

  • Computer Research Institute of Montreal

Research output: Contribution to Book/Report types › Contribution to conference proceedings › peer-review

10 Citations (Scopus)

Abstract

Deaf and hearing-impaired people capture information in video through visual content and captions. Those activities require different visual attention strategies and up to now, little is known on how caption readers balance these two visual attention demands. Understanding these strategies could suggest more efficient ways of producing captions. Eye tracking and attention overload detections are used to study these strategies. Eye tracking is monitored using a pupilcenter-corneal-reflection apparatus. Afterward, gaze fixation is analyzed for each region of interest such as caption area, high motion areas and faces location. This data is also used to identify the scanpaths. The collected data is used to establish specifications for caption adaptation approach based on the location of visual action and presence of character faces This approach is implemented in a computer-assisted captioning software which uses a face detector and a motion detection algorithm based on the Lukas-Kanade optical flow algorithm. The different scanpaths obtained among the subjects provide us with alternatives for conflicting caption positioning. This implementation is now undergomg a user evaluation with hearing impaired participants to validate the efficiency of our approach.

Original languageEnglish
Title of host publicationProceedings of SPIE-IS and T Electronic Imaging - Human Vision and Electronic Imaging XII
PublisherSPIE
ISBN (Print)0819466050, 9780819466051
DOIs
Publication statusPublished - 2007
Externally publishedYes
Event12th Human Vision and Electronic Imaging, HEVI 2007 - San Jose, CA, United States
Duration: 29 Jan 2007 → 1 Feb 2007

Publication series

NameProceedings of SPIE - The International Society for Optical Engineering
Volume6492
ISSN (Print)0277-786X

Conference

Conference12th Human Vision and Electronic Imaging, HEVI 2007
Country/TerritoryUnited States
CitySan Jose, CA
Period29/01/07 → 1/02/07

Keywords

  • Captioning systems
  • Eye tracking
  • Human-computer interaction
  • Motion detection

ASJC Scopus subject areas

  • Electronic, Optical and Magnetic Materials
  • Instrumentation
  • Condensed Matter Physics
  • Computer Science Applications
  • Applied Mathematics
  • Electrical and Electronic Engineering

Fingerprint

Dive into the research topics of 'Improving video captioning for deaf and hearing-impaired people based on eye movement and attention overload'. These topics are generated from the title and abstract of the publication. Together, they form a unique fingerprint.

Cite this