PhD student wins award for paper on using distance information to improve sound separation
PhD student wins award for paper on using distance information to improve sound separation
Rajesh Rameshbabu, a PhD student in Assistant Professor Ryan M. Corey’s Listening Technology Lab, received a Student Paper Award from the Acoustical Society of America’s Proceedings of Meetings on Acoustics journal at their most recent annual gathering. His work was also featured on the society’s official podcast in August. The winning paper, “Learning distance-dependent spatial structure for multichannel source separation,” relates to distance-based sound separation.
The goal of his work is to make it easier for people to hear one another in noisy spaces such as restaurants by enhancing nearby sounds and removing sounds that are far away. This technology can also be useful in other devices such as smart speakers, conferencing systems, and other devices that capture sound in noisy environments.
Rameshbabu looked at ways to use distance information to improve the performance of AI-based sound separation algorithms, which is a step toward full distance-based separation.
Rameshbabu earned his MS from the Indian Institute of Technology Mandi, where he researched music source separation and how machines can understand complex acoustic scenes. His research at UIC has expanded into multi-channel source separation, interference reduction in live recordings, distance-based source separation, and distributed acoustic sensor networks.
“I am fascinated by how the physics of sound, like distance, room acoustics, and microphone geometry, can be combined with machine learning to build audio systems that are not only more robust but that better reflect how sound behaves in real world,” Rameshbabu said.
Humans can distinguish individual speakers in a conversation or pick up different instruments in a musical composition even in noisy environments. He hopes to imbue machines with the same ability.
Traditional hearing aids amplify everything, making conversation in noisy environments even harder to understand. But by separating out sounds, conversation becomes much clearer. Current microphones can’t determine individual sources—sounds overlap in time and frequency. Instead of separating sounds the traditional way, based on who is producing them, Rameshbabu aims to separate sounds by how far away each one is to the microphone.
Just as someone’s voice becomes quieter as they walk far away from the listener, the sound source reaching from the microphone also becomes weaker.
“The distance also changes the spectral balance–higher frequencies are absorbed more easily by the air and by their interaction with the room other than the low frequencies,” Rameshbabu said. “So as the source moves far away, the sound gradually loses some of its high-frequency content and becomes slightly more muffled.”
Using a neural network, he and Corey’s system, DistanceNet, learns how all of the acoustic cues work together to understand distance, and home in on what the listener wants to hear. Over time, the system learns an “embedding” or summary of what the network has learned. Not only will the system detect how close the sound is but it will also capture how distance has shaped the sound through loudness, reverberation, and other acoustic cues.
Rameshbabu hopes to evaluate the framework in realistic acoustic environments, and eventually extend the work beyond two-source, near-far scenarios.
Listen to the podcast episode with Across Acoustics in its entirety.