title   
  

Audio-visual speech recognition with background music using single-channel source separation

Grais, Emad Mounir and Topkaya, İbrahim Saygın and Erdoğan, Hakan (2012) Audio-visual speech recognition with background music using single-channel source separation. In: 20th IEEE Conference on Signal Processing and Communications Applications (SIU2012), Mugla, Turkey

[img]
Preview
PDF - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader
230Kb

Official URL: http://dx.doi.org/10.1109/SIU.2012.6204436

Abstract

In this paper, we consider audio-visual speech recognition with background music. The proposed algorithm is an integration of audio-visual speech recognition and single channel source separation (SCSS). We apply the proposed algorithm to recognize spoken speech that is mixed with music signals. First, the SCSS algorithm based on nonnegative matrix factorization (NMF) and spectral masks is used to separate the audio speech signal from the background music in magnitude spectral domain. After speech audio is separated from music, regular audio-visual speech recognition (AVSR) is employed using multi-stream hidden Markov models. Employing two approaches together, we try to improve recognition accuracy by both processing the audio signal with SCSS and supporting the recognition task with visual information. Experimental results show that combining audio-visual speech recognition with source separation gives remarkable improvements in the accuracy of the speech recognition system.

Item Type:Papers in Conference Proceedings
Subjects:T Technology > T Technology (General)
ID Code:19010
Deposited By:Emad Mounir Grais
Deposited On:16 May 2012 14:27
Last Modified:28 Feb 2014 15:55

Repository Staff Only: item control page