Creating more efficient data centers for AI

Prof. Lingjia Tang has earned an NSF CAREER Award for her proposal to develop software that can increase data center efficiency. The project, “QoS-aware Systems for Accelerated Datacenters,” will address the challenge of providing enough efficient infrastructure to meet the increasing computational demands of Artificial Intelligence (AI) and Machine Learning (ML) applications.

To support the ever-growing prevalence of ML and AI applications, data centers are turning to specialized hardware accelerators for computation that offer up to 100x improvement in efficiency over traditional CPU infrastructures. However, while the community has largely focused on hardware designs, there is very little research attention on designing system software that can manage the performance and efficiency of hardware accelerated data centers. Current data center system software is mostly tailored to CPU platforms and lacks the ability to take full advantage of the accelerators.

The goal of Tang’s project is to redesign data center systems to support large-scale use of these accelerators and meet the future computational demand. Tang’s system will schedule and allocate resources among mixed platforms composed of both general-purpose processors and different accelerators.

Tang will focus on three main goals. First, she’ll design the data center scheduler for these mixed, accelerated infrastructures to predict the performance interference of the accelerators and maximize their use. Second, she’ll design an application-level acceleration manager to identify bottlenecks in the data center, estimate potential improvements using different accelerators, and dynamically allocates accelerator resources. And third, she’ll design a resource manager for accelerated platforms that manages unique resource sharing and reconfiguration behaviors on accelerators.

Tang has spent her career making computers more conversational, as well as democratizing the smart tools typically held close by big enterprises. Products of her work include Sirius, an application that took the concept of the voice-controlled virtual assistant (like Siri or Cortana) and made it open-source, and Adasa, an in-vehicle digital assistant that responds to drivers’ questions and commands in natural language. As Chief Operating Officer and Co-Founder of AI startup Clinc, she helped develop the Clinc Platform that democratizes conversational AI technology and allows any member of an organization to design and deploy conversational tools catered to the organization’s needs. One of Clinc’s own standalone products is Finie, the “Siri of personal banking,” which helps bank customers talk to their accounts in a natural and conversational way.

About the NSF CAREER Award

The CAREER grant is one of the National Science Foundation’s most prestigious awards, conferred for “the early career-development activities of those teacher-scholars who most effectively integrate research and education within the context of the mission of their organization.”