GAN-Based Multi-Microphone Spatial Target Speaker Extraction

Abstract

Spatial target speaker extraction isolates a desired speaker’s voice in multi-speaker environments using spatial cues, such as the \ac{DoA}. Although recent \ac{DNN}-based discriminative methods have shown significant performance improvements, the potential of generative approaches, such as generative adversarial networks (GANs), remains largely unexplored for this problem. In this work, we demonstrate that a GAN can effectively leverage both noisy mixtures and spatial information to extract and generate the target speaker’s speech. By conditioning the GAN on intermediate features of a discriminative spatial filtering model in addition to \ac{DoA}, we enable steerable target extraction with high spatial resolution of 5$^\circ$, outperforming state-of-the-art discriminative methods.

Evaluation Scenarios

In our work, we evaluate our proposed method with different SOTA generative and discriminative deep learning-based noise reduction methods in various SNR scenraios.

Following you can find some processed samples with different Methods:

1. Low SNR Dataset --> Go to the samples

2. Real Recordings --> Will be added soon

2. DNS Challenge No-Reverb --> Will be added soon

1. Low SNR Dataset

Item 1 (Speaker: Female)

Item 2 (Speaker: FeMale)

Item 3 (Speaker: Female)

Item 4 (Speaker: FeMale)

Item 5 (Speaker: Male)

Item 6 (Speaker: Male)

Item 7 (Speaker: Male)

Item 8 (Speaker: Male)

Item 9 (Speaker: Male)

Item 10 (Speaker: Male)