Using Jupyter at scale within an enterprise: 10,000 users, 10,000 notebooks

Jupyter Berkeley Seminar


September 27, 2019
1:00pm to 2:00pm
1011 Evans Hall
Get Directions

The National Security Agency (NSA) has a large use-case for Jupyter. While traditionally known as a data science platform, we have worked to expand Jupyter's use within our organization to include our large user-base of intelligence analysts. These analysts, hard at work culling and correlating information from a multitude of repositories, are the domain experts who are closest to the analytic challenges we face. Jupyter has empowered this community - who do not traditionally come from software engineering backgrounds - to translate their tradecraft into code, making that tradecraft more reproducible, more efficient, and more sharable.

The scale of our effort has forced us to address a number of challenges. With over a thousand Python authors, ten thousand notebooks, and ten thousand Jupyter users, we developed tools and approaches to manage, curate, and sustain crowd-sourced development of Jupyter notebook based analytics. We identified appropriate training paths to introduce Python and Jupyter into communities that most often lack prior backgrounds in coding. We outlined common use-cases for Jupyter, and worked to lower the barrier to entry to make the platform as approachable as possible. Most importantly, we ensured that all users of Jupyter within our Agency meet our strict data security and access controls.

This talk sheds light upon the secretive world of the NSA and shares some unique insight from our experience of adopting Jupyter across a large organization. We will also detail how we are working to support the open source community surrounding Jupyter.


Dave Stuart

US Department of Defense

Dave Stuart is a senior technical executive within the US Department of Defense where he has worked for the last 15 years. He is currently leading a large-scale effort to transform the workflows of thousands of enterprise business analysts through Jupyter and Python adoption, making their tradecraft more efficient, sharable, and repeatable. Previously, Dave led multiple grass-roots technology adoption efforts, developing innovative training methods that tangibly increased the technical proficiency of a large non-coding enterprise workforce.