Monaural Singing Voice Separation by Non-negative Matrix Partial Co-Factorization with Temporal Continuity and Sparsity Criteria

Separating singing voice from music accompaniment for monaural recordings is very useful in many applications, such as lyrics recognition and singer identification. Based on non-negative matrix partial co-factorization (NMPCF), we propose an improved algorithm which restricts the activation coefficients of singing voice components to be temporal continuous and sparse in each frame. Temporal continuity is favored by using a cost term which is the sum of squared difference between the activation coefficients in adjacent frames, and sparsity is favored by penalizing nonzero values for each frame. For the separated singing voice, we quantify the performance of the system by the signal-to-noise ratio (SNR) gain and the accuracy of singer identification. The experiments show that the constraints of temporal continuity and sparsity criteria both can improve the performance of singing voice separation, especially the constraint of temporal continuity.