Real-time Ultrasound-enhanced Multimodal Imaging of Tongue using 3D Printable Stabilizer System: A Deep Learning Approach

Authors

  • M. Hamed Mozaffari University of Ottawa http://orcid.org/0000-0002-2297-6114
  • Won-Sook Lee Professor, School of Electrical Engineering and Computer Science (EECS), University of Ottawa, Ottawa, Ontario, Canada, K1N 6N5

Abstract

Despite renewed awareness of the importance of articulation, it remains a challenge for instructors to handle the pronunciation needs of language learners. There are relatively scarce pedagogical tools for pronunciation teaching and learning. Unlike inefficient, traditional pronunciation instructions like listening and repeating, electronic visual feedback (EVF) systems such as ultrasound technology have been employed in new approaches.
Recently, an ultrasound-enhanced multimodal method has been developed for visualizing tongue movements of a language learner overlaid on face-side of the speaker's head. That system was evaluated for several language courses via a blended learning paradigm at the university level.
The result was asserted that visualizing articulator’s system as biofeedback to language learners will significantly improve articulation learning efficiency.
In spite of that successful usage of multimodal technique for pronunciation training, it still requires manual works and human manipulation. In this article, we aim to contribute to this growing body of research by addressing difficulties of the previous approaches by proposing a new comprehensive, automatic, real-time multimodal pronunciation training system, benefits from powerful artificial intelligence techniques.
The main objective of this research was to combine the advantages of ultrasound technology, three-dimensional printing, and deep learning algorithms to enhance the performance of previous systems. Our preliminary pedagogical evaluation of the proposed system revealed a significant improvement in flexibility, control, robustness, and autonomy.

Author Biography

M. Hamed Mozaffari, University of Ottawa

School of Electrical Engineering and Computer Science

Additional Files

Published

2020-03-17

How to Cite

1.
Mozaffari MH, Lee W-S. Real-time Ultrasound-enhanced Multimodal Imaging of Tongue using 3D Printable Stabilizer System: A Deep Learning Approach. Canadian Acoustics [Internet]. 2020 Mar. 17 [cited 2024 Nov. 22];48(1). Available from: https://jcaa.caa-aca.ca/index.php/jcaa/article/view/3305

Issue

Vol. 48 No. 1 (2020)

Section

Article - Speech Sciences

License

Author Licensing Addendum

This Licensing Addendum ("Addendum") is entered into between the undersigned Author(s) and Canadian Acoustics journal published by the Canadian Acoustical Association (hereinafter referred to as the "Publisher"). The Author(s) and the Publisher agree as follows:

  1. Retained Rights: The Author(s) retain(s) the following rights:

    • The right to reproduce, distribute, and publicly display the Work on the Author's personal website or the website of the Author's institution.
    • The right to use the Work in the Author's teaching activities and presentations.
    • The right to include the Work in a compilation for the Author's personal use, not for sale.

  2. Grant of License: The Author(s) grant(s) to the Publisher a worldwide exclusive license to publish, reproduce, distribute, and display the Work in Canadian Acoustics and any other formats and media deemed appropriate by the Publisher.

  3. Attribution: The Publisher agrees to include proper attribution to the Author(s) in all publications and reproductions of the Work.

  4. No Conflict: This Addendum is intended to be in harmony with, and not in conflict with, the terms and conditions of the original agreement entered into between the Author(s) and the Publisher.

  5. Copyright Clause: Copyright on articles is held by the Author(s). The corresponding Author has the right to grant on behalf of all Authors and does grant on behalf of all Authors, a worldwide exclusive license to the Publisher and its licensees in perpetuity, in all forms, formats, and media (whether known now or created in the future), including but not limited to the rights to publish, reproduce, distribute, display, store, translate, create adaptations, reprints, include within collections, and create summaries, extracts, and/or abstracts of the Contribution.