Non-negative Matrix Factorization on a Multi-lingual Overlapped Speech Signal: A Signal and Perception Level Analysis

Abstract

A complex acoustic scenario comprising overlapping speeches from multiple speakers in the presence of noise renders speech recognition perform poorly in hands-free devices. This scenario turns out to be more complex in India, a country where 96.71% of the population speaks one of the 22 scheduled languages. Therefore, an audio source separation algorithm that mitigates the interference from other speakers and effectively enhances the articulacy and quality of source speech may be added as a pre-processor in speech recognition systems. This research, therefore, investigates the non-negative matrix factorization (NMF) algorithm's effectiveness for the separation of source in an overlapping multi-lingual multi-dialect single-channel speech mixture scenario, an inherent characteristic of a cocktail party problem in India. The objective is to analyze the signal level metrics and perception level metrics of a speech source-separated from a multi-lingual overlapped speech signal. The languages used for the same are English and two Indo-Aryan languages, Marathi and Bengali. One of the experimental results demonstrated that the source to distortion ratio (SDR) of separated target source from English-Bengali and English-Marathi speech mixture is 0.4 and 1.3 dB higher than English-English speech mixed signals, respectively. Therefore, the experiments highlight an improvement in separating sources from mixed speech signals with different language combinations than the same language.