PyWren: Pushing Microservices to Teraflops
Much of cloud computing infrastructure remains hard to use in spite of decades of both academic research and commercialization. Fortunately, recent technologies developed for web services and internet startups can be repurposed to enable a much lower-friction scalable cloud experience. Our goal is making the power, elasticity, and dynamism of commercial cloud services like Amazon's EC2 accessible to busy applied physicists, electrical engineers, and data scientists as well as a compelling new capability over Matlab, hopefully encouraging migration. We built PyWren, a transparent distributed execution engine on top of AWS Lambda, which hopefully simplifies many scale-out use cases for data science and computational imaging. We will demo applications built on our framework and seek user input into next directions.
Joint work with Shivaram Venkataraman, Qifan Pu, Ion Stoica, and Ben Recht.
Speaker Bio: Eric Jonas is an exhausted postdoc working in EECS with Ben Recht on machine learning and computational acquisition in the new Berkeley Center for Computational Imaging.