Dhruv Jain earns dissertation award for work on sound accessibility

This story was written by Roger Van Scyoc for the University of Washington’s Paul G. Allen School of Computer Science and Engineering

The University of Washington Allen School of Computer Science has recognized assistant professor Dhruv Jain with the William Chan Memorial Dissertation Award, which honors graduate dissertations of exceptional merit and is named in memory of the school’s late graduate student William Chan. Jain was chosen for his work in advancing new sound awareness systems for accessibility.

Jain’s dissertation, titled “Sound Sensing and Feedback Techniques for Deaf and Hard of Hearing People,” investigated the creation and use of several sound awareness systems in addition to exploring how d/Deaf and hard-of-hearing (DHH) individuals feel about emerging sound awareness technology. One of the systems included HoloSound, an augmented reality system that provides real-time captioning, sound identity and sound location information to DHH users via a wearable device, as well as HomeSound, an Internet-of-Things system that integrates smart displays throughout the home to sense common sounds and produce a single visualization of sound activity within the household. Jain also led the team that developed SoundWatch, a smartwatch app that provides DHH individuals with better awareness of incoming sounds. 

“Beyond accessibility, the technical innovations in the field of sound sensing and feedback proposed in the thesis have wide applications for other high-impact domains,” Jain said. “Some of which include ecological surveys, home-automation, game audio debugging and appliance repairs.”

Allen School professor Jon E. Froehlich and Human Centered Design & Engineering professor and Allen School adjunct professor Leah Findlater co-advised Jain, whose work in the Makeability Lab helped facilitate sound accessibility through systems employing human computer interaction (HCI) and artificial intelligence (AI).

“Dhruv’s dissertation research makes fundamental advances in the design of sound sensing and feedback systems for people who are deaf or hard of hearing,” Froehlich said. “Throughout his dissertation work, Dhruv has worked closely with the DHH community to understand diverse needs and evaluate his systems, including through large online surveys, interviews and field deployments.”

Jain’s own experiences as a DHH individual informed his research and helped shape his focus on the user experience. 

“Dhruv’s dissertation not only exemplifies the human-centered design process in the creation of accessible technologies but also makes transformative technical innovations in integrating AI and HCI to improve information access,” Froehlich added. “As a testament to its impact, his work has received multiple paper awards and a Microsoft Research Dissertation Grant, and SoundWatch has been released and downloaded by over 2,000 Android watch users worldwide.”

Jain graduated in July and joined CSE as a professor in September. He is also affiliated with the U-M School of Information and Department of Family Medicine.

“I am immensely grateful to the Allen School staff and faculty in supporting me throughout my research journey,” Jain said. “Especially my advisors Jon Froehlich and Leah Findlater, committee members Jennifer Mankoff, Jacob Wobbrock and Richard Ladner, and staff members Elise Dorough, Elle Brown, Emma Gebben, Sandy Kaplan, Aaron Timss, Hector Rodriguez and Chiemi Yamaoka.”