Rada Mihalcea to study physiological and linguistic signals of human behavior

Prof. Rada Mihalcea is co-PI for a new two-year grant from the National Science Foundation that will explore a new generation of computational tools for joint modeling of physiological and linguistic signals of human behavior with a focus on deception detection. The project is an interdisciplinary collaboration between Prof. Mihalcea and Prof. Mihai Burzo (PI) of Mechanical Engineering at U-M Flint.

Work on deception has received significant attention from several fields of study, ranging from
physiology to psychology, sociology, linguistics, and computer vision. While a number of studies
have been carried out in the past for the automatic detection of deception, most of this previous
work has primarily targeted one modality at a time. There is very little work that has considered
the simultaneous exploitation of multiple modalities for the purpose of deception detection.

This work will be the first to create joint physiological and linguistic models for the recognition
of deception. The researchers are pursuing three lines of inquiry: First, a novel physio-linguistic dataset of deceit is built, covering several different domains. Second, rule-based classifiers for deception detection are explored, using physiological features (e.g., heart rate, respiration rate, galvanic skin response, skin temperature), as well as linguistic features. Third, data-driven learning approaches for multimodal deception detection are developed, taking advantage of the recent progress in early, late, and temporal fusion models.

The project is exploratory in nature, and acts as a catalyst for novel research problems. First, it explores rich sets of multimodal features extracted from physiological and linguistic modalities, analyzing their effectiveness in the recognition of deceit. Second, it also explores the integration of multiple physio-linguistic modalities, through experiments with rule-based and data-driven techniques that fuse multimodal features into joint deception analysis models. To address the challenges of multimodal research work, the team working on this project brings together experts from the fields of bio-sensors, computational linguistics, and physiology and behavioral sciences.

The project has high potential payoffs, as models of deception detection have broad applicability, including: the development of critical tools for various applications in fields such as criminal justice, intelligence, and security; the enhancement of applications that can be negatively affected by the presence of deceit, such as opinion analysis or modeling of human communication; and a deeper understanding of fundamental aspects of human behavior, which can positively impact medical applications in psychiatry and psychology. The tools and datasets produced during this project will be made freely available for the research community.

Prof. Rada Mihalcea received a PhD in Computer Science and Engineering from Southern Methodist University and a PhD in Linguistics from Oxford University. She joined the faculty at Michigan in 2013 after serving on the faculty at the Department of Computer Science at the University of North Texas.

Prof. Mihalcea’s research interests are in computational linguistics, with a focus on lexical semantics, graph-based algorithms for natural language processing, and multilingual natural language processing. She is currently involved in a number of research projects, including word sense disambiguation, monolingual and cross-lingual semantic similarity, subjectivity, sentiment, and emotion analysis, multimodal affect analysis, and computational humor.

She is the recipient of a NSF CAREER award and a Presidential Early Career Award for Scientists and Engineers. In 2013, she was made an honorary citizen of her hometown of Cluj-Napoca, Romania.