Principles for data analysis workflows

Sara Stoudt , Váleri N. Vásquez, Ciera C. Martinez

PLOS Computational Biology
March 18, 2021

Abstract: A systematic and reproducible “workflow”—the process that moves a scientific investigation from raw data to coherent research question to insightful contribution—should be a fundamental part of academic data-intensive research practice. In this paper, we elaborate basic principles of a reproducible data analysis workflow by defining 3 phases: the Explore, Refine, and Produce Phases. Each phase is roughly centered around the audience to whom research decisions, methodologies, and results are being immediately communicated. Importantly, each phase can also give rise to a number of research products beyond traditional academic publications. Where relevant, we draw analogies between design principles and established practice in software development. The guidance provided here is not intended to be a strict rulebook; rather, the suggestions for practices and tools to advance reproducible, sound data-intensive analysis may furnish support for both students new to research and current researchers who are new to data-intensive work.

Featured Fellows

Sara Stoudt

Statistics, UC Berkeley
Alumni - BIDS Data Science Fellow

Váleri N. Vásquez

Energy and Resources Group
BIDS Data Science Fellow

Ciera Martinez

Biodiversity and Environmental Sciences Lead