Sound-Stream Ii: Towards Real-Time Gesture-Controlled Articulatory Sound Synthesis

Authors

  • Pramit Saha University of British Columbia
  • Debasish Ray Mohapatra University of British Columbia
  • Praneeth SV University of British Columbia
  • Sidney Fels University of British Columbia

Keywords:

Articulatory speech synthesizer, voice controller, intrinsic and extrinsic tongue muscles, force-based controller, gesture-to-muscle activation, Sound Stream

Abstract

We present an interface involving four degrees-of-freedom (DOF) mechanical control of a two dimensional, mid-sagittal tongue through a biomechanical toolkit called ArtiSynth and a sound synthesis engine called JASS towards artic-ulatory sound synthesis. As a demonstration of the project, the user will learn to produce a range of JASS vocal sounds, by varying the shape and position of the ArtiSynth tongue in 2D space through a set of four force-based sensors. In otherwords, the user will be able to physically play around with these four sensors, thereby virtually controlling the magnitude of four selected muscle excitations of the tongue to vary articulatory structure. This variation is computed in terms of ‘Area Functions’ in ArtiSynth environment and communicated to the JASS based audio-synthesizer coupled with two-mass glottal excitation model to complete this end-to-end gesture-to-sound mapping.

Author Biography

Pramit Saha, University of British Columbia

Research Assistant, Human Communication Technologies Lab,

Department of Electrical and Computer Engineering

Additional Files

Published

2019-02-21

How to Cite

1.
Saha P, Mohapatra DR, SV P, Fels S. Sound-Stream Ii: Towards Real-Time Gesture-Controlled Articulatory Sound Synthesis. Canadian Acoustics [Internet]. 2019 Feb. 21 [cited 2024 Nov. 23];46(4):58-9. Available from: https://jcaa.caa-aca.ca/index.php/jcaa/article/view/3248

Issue

Vol. 46 No. 4 (2018)

Section

Proceedings of the Acoustics Week in Canada

License

Author Licensing Addendum

This Licensing Addendum ("Addendum") is entered into between the undersigned Author(s) and Canadian Acoustics journal published by the Canadian Acoustical Association (hereinafter referred to as the "Publisher"). The Author(s) and the Publisher agree as follows:

  1. Retained Rights: The Author(s) retain(s) the following rights:

    • The right to reproduce, distribute, and publicly display the Work on the Author's personal website or the website of the Author's institution.
    • The right to use the Work in the Author's teaching activities and presentations.
    • The right to include the Work in a compilation for the Author's personal use, not for sale.

  2. Grant of License: The Author(s) grant(s) to the Publisher a worldwide exclusive license to publish, reproduce, distribute, and display the Work in Canadian Acoustics and any other formats and media deemed appropriate by the Publisher.

  3. Attribution: The Publisher agrees to include proper attribution to the Author(s) in all publications and reproductions of the Work.

  4. No Conflict: This Addendum is intended to be in harmony with, and not in conflict with, the terms and conditions of the original agreement entered into between the Author(s) and the Publisher.

  5. Copyright Clause: Copyright on articles is held by the Author(s). The corresponding Author has the right to grant on behalf of all Authors and does grant on behalf of all Authors, a worldwide exclusive license to the Publisher and its licensees in perpetuity, in all forms, formats, and media (whether known now or created in the future), including but not limited to the rights to publish, reproduce, distribute, display, store, translate, create adaptations, reprints, include within collections, and create summaries, extracts, and/or abstracts of the Contribution.