Sensorial substitution system with encoding of visual objects into sounds

Authors

  • Damien Lescal NECOTIS, Dept. GEGI, Université de Sherbrooke, Quebec, QC J1K 2R1, Canada
  • Jean Rouat NECOTIS, Dept. GEGI, Université de Sherbrooke, Quebec, QC J1K 2R1, Canada
  • Stéphane Molotchnikoft Department de Sciences Biologiques, Université de Montréal, Quebec, QC H3C 3J7, Canada

Keywords:

Auditory stimuli, Complex sounds, Information carriers, Mental representations, Receptive fields, Substitution systems, Visual objects, Visual scene

Abstract

Visual and auditory prostheses involve surgeries that are complex, expensive, and invasive. They are limited to a small number of electrodes and can only be used when the impairment is peripheral. The Vibe and PSVA encode the entire image in one complex sound. The PSVA uses frequencies that are associated with each pixel and increase from left to right and from bottom to top of the image. The Vibe splits the image into several regions that are equivalent to receptive fields. The challenge in this project resides in the design of a suitable encoding of the visual scene into auditory stimuli such that the content of the sound carries the most important characteristics of the visual scene. These sounds should be shaped in a way that the subject can build mental representations of visual scenes even if the information carrier is the auditory pathway. The complex sound is the sum of all single sounds from each segment. One complex sound is generated for the right ear and another one for the left.

Additional Files

Published

2011-09-01

How to Cite

1.
Lescal D, Rouat J, Molotchnikoft S. Sensorial substitution system with encoding of visual objects into sounds. Canadian Acoustics [Internet]. 2011 Sep. 1 [cited 2025 Feb. 21];39(3):106-7. Available from: https://jcaa.caa-aca.ca/index.php/jcaa/article/view/2433

Issue

Vol. 39 No. 3 (2011)

Section

Proceedings of the Acoustics Week in Canada

License

Author Licensing Addendum

This Licensing Addendum ("Addendum") is entered into between the undersigned Author(s) and Canadian Acoustics journal published by the Canadian Acoustical Association (hereinafter referred to as the "Publisher"). The Author(s) and the Publisher agree as follows:

  1. Retained Rights: The Author(s) retain(s) the following rights:

    • The right to reproduce, distribute, and publicly display the Work on the Author's personal website or the website of the Author's institution.
    • The right to use the Work in the Author's teaching activities and presentations.
    • The right to include the Work in a compilation for the Author's personal use, not for sale.

  2. Grant of License: The Author(s) grant(s) to the Publisher a worldwide exclusive license to publish, reproduce, distribute, and display the Work in Canadian Acoustics and any other formats and media deemed appropriate by the Publisher.

  3. Attribution: The Publisher agrees to include proper attribution to the Author(s) in all publications and reproductions of the Work.

  4. No Conflict: This Addendum is intended to be in harmony with, and not in conflict with, the terms and conditions of the original agreement entered into between the Author(s) and the Publisher.

  5. Copyright Clause: Copyright on articles is held by the Author(s). The corresponding Author has the right to grant on behalf of all Authors and does grant on behalf of all Authors, a worldwide exclusive license to the Publisher and its licensees in perpetuity, in all forms, formats, and media (whether known now or created in the future), including but not limited to the rights to publish, reproduce, distribute, display, store, translate, create adaptations, reprints, include within collections, and create summaries, extracts, and/or abstracts of the Contribution.