With so many movies, TV shows, and content coming out every year, analyzing media can seem daunting. However, with computational methods, new insights can be uncovered, but what exactly to measure is an open question. As a part of Cultural Analytics workshop week, Lauren Tilton, Professor of Liberal Arts and Digital Humanities from the University of Richmond in Virginia, presented her research on the use of computational methods to analyze film and television, a term she is calling “multimodal cinemetrics.” Along with collaborator Taylor Arnold, Tilton directs the Distant Viewing Lab to apply computational methods to digital media on a large scale.
Tilton’s research focuses on questions such as determining proper units of analysis when studying time-based media, and how a multimodal approach may enable greater insights. Through exploring different theories of media, historical research, and considering the meaning of “cinema” itself, Tilton inspects the concept of multimodal cinemetrics.
Historical and Research Background
Tilton outlines the context of media studies by introducing different researchers and their ideas about film, including a film’s purpose, techniques of editing, and narrative flow. She referenced major film theorists and filmmakers who developed key concepts such as a “scene” and “long shot.” Additionally, Tilton introduced multiple researchers of television theory, who explored editing styles, narrative television, and the shifts of television in the network era. Classifying television can be a complicated endeavor, with traditional categorization along elements such as genre failing to capture other elements, such as prestige or “cinematic” television. By providing this backdrop, Tilton frames the conversation around the way time-based media has historically been analyzed, and how computational techniques are influenced by the underlying theories that define what is measured.
Photo: Tilton introducing “Multimodal Cinemetrics”
Quantifying the Analysis
An early researcher, Barry Salt, approached the subject of quantification of film from a multimodal perspective, incorporating visual shots and audio in his analysis. The idea of cinemetrics emerged from Salt’s work. The group focused on measures such as average shot length, and on director style and changes in pacing over time. Later researchers started extending multimodality. Tilton returned to Jeremy Butler, who used sound, specifically laugh tracks, in his analysis. Furthermore, Charles O’Brien tracked the evolution from silent to sound based films. He used different metrics, classifying shots as action, dialogue, or singing shots. These metrics and measures illustrate how the first step in applying computational methods is determining what to measure.
Metrics for Computational Analysis today
Finally, as a part of the Distant Viewing Lab, Tilton and Arnold are exploring how to theorize large scale analysis of images using machine learning. This analysis has led them to consider new questions, such as how to define a “shot.” Some may define a shot based on the length of time it covers, while others define it by the scales or closeness of the subject. Classifying shots by just a single feature is difficult, and a new approach that categorizes features differently could capture deeper complexities and enable further analysis.
Photo: Tilton explaining Distant Viewing Lab’s work
Language and Meaning
One challenge in this field is “finding language or words… that reflect the cultural, historical, and computational relations” that are used in analysis. For example, Tilton covered the challenges of defining what cinema even is. The word “cinema” comes from a term meaning movement, but the cultural meaning of cinema has evolved over time. Cinema eventually came to refer to a theater, and now the meaning is shifting again with the usage of the word differing on streaming platforms.
Furthermore, the structures of film and television today are blurring, such as the rise of the “mini-series” that are similar to films. Finally, the rise of streaming platforms has impacted the way media is released and analyzed. All of these factors play a part when considering how to analyze media.
Reflections
Tilton’s talk highlighted the intersection of humanities research and computational methods, and the ways that culture shapes how computational methods are applied to analysis. Multimodal approaches can unlock new insights previously hidden. Through conversations such as these, the BIDS community learns from each other, explores complex topics, and fosters new learnings.
Find the recording of the full talk on the I School website here. To stay in touch and join conversations of critical cultural importance, please join our Cultural Analytics mailing list by visiting this page or emailing bids-cultural-analytics+subscribe@lists.berkeley.edu