Computational Methods for Machine Learning and Artificial Intelligence

BIDS Machine Learning and Science Forum
Date: Monday, March 7, 2022
Time: 11:00 AM - 12:00 PM Pacific Time
Location: Participate remotely using this Zoom link 

Computational Methods for Machine Learning and Artificial Intelligence

Speaker: Somayeh Sojoudi, UC Berkeley
Abstract: The area of data science needs efficient computational methods with provable guarantees that can cope with complex nature and the high nonlinearity of many real-world systems. Practitioners often design heuristic algorithms tailored to specific applications, but the theoretical underpinnings of these methods remain a mystery and this limits their usage in safety-critical systems. In this talk, we aim to address the above issue for some machine learning problems. First, we study the problem of certifying the robustness of neural networks against adversarial inputs. To diminish the relaxation error caused by popular linear programming and semidefinite programming certification methods, we propose partitioning the input uncertainty set and show that this approach reduces the relaxation error. We then develop a practical partitioning technique for large-scale networks. We also study the problem of robust neural network training and develop convex formulations to train networks that are robust to adversarial inputs, followed by efficient training algorithms with global convergence guarantees. In order to accelerate the computation, there is a major effort in the machine learning community to understand when simple local search algorithms could solve nonlinear problems to global optimality. A key proof technique relies on the notion of Restricted Isometry Property, whose conservatism is not well understood and cannot be applied to nonsmooth problems either. We discuss our recent results on addressing these problems. In particular, we introduce the notion of “global functions”, as a major generalization of convex functions, which allows us to study the non-existence of spurious local minima for nonconvex and nonsmooth learning problems. We demonstrate the results on tensor decomposition with outliers, video processing, and online optimization in machine learning.

The BIDS Machine Learning and Science Forum meets biweekly to discuss current applications across a wide variety of research domains in the physical sciences and beyond. Hosted by BIDS Affiliates Uroš Seljak (professor of Physics at UC Berkeley) and Ben Nachman (physicist at Lawrence Berkeley National Laboratory), these active sessions bring together domain scientists, statisticians, and computer scientists who are either developing state-of-the-art methods or are interested in applying these methods in their research.  To receive email notifications about upcoming meetings, or to request more information, please contact the organizers at berkeleymlforum@gmail.com. All interested members of the UC Berkeley and Berkeley Lab communities are welcome and encouraged to attend.