Linguistic-Mediation of Visual Saliency: Using ROC to Compare Multi-Dimensional Datasets

Lecture

Speaker(s)

Stephanie Huette

Assistant Professor, University of Memphis

Dr. Huette received a BS in Psychology from the University of Iowa and a PhD in Cognitive and Information Sciences from the University of California, Merced. She is an assistant professor of psychology at the University of Memphis and the PI of the Language and Behavior Lab. Her work focuses on using real-time behavioral measures, such as eye-tracking, to examine the mechanisms of language processing in natural contexts and how pragmatics interact with these mechanisms at various timescales to produce comprehension in the moment and form representations of meaning over time.

Dr. Huette is also affiliated with the Institute for Intelligent Systems at the University of Memphis and the Center for Climate Communication at the University of California, Merced.

Website: stephaniehuette.com

Friday, April 21, 2017

BIDS Data Science Lecture Series | April 21, 2017 | 1:10-2:30 p.m. | 190 Doe Library, UC Berkeley

Models of visual saliency have been used for decades to predict eye movements, in many cases very accurately (Itti, Koch, & Niebur, 1998). However, eye movements change as a function of task (Yarbus, 1967), which often includes instruction or questions delivered through speech or text.There are an infinite number of combinations of visual scenes and linguistic input, so characterizing every situation is impossible. Instead, principles of how language might mediate the salience of an image should be developed so more dynamic, interactive models of eye-movement patterns can be established. Two experiments, receiver operating characteristics (ROC) analyses, and potential applications to other areas that work with multi-dimensional datasets will be discussed. The ROC analysis is useful for evaluating data transformations, the accuracy of classification algorithms, individual or strategic differences in human-generated data, and much more.

Share this video