Abstract: The ability to understand physical interaction among objects lies at the core of human cognition; it is also essential in building intelligent machines that see and manipulate objects in the real world. In this talk, I'll present our recent work on using deep learning to approximate physical interaction, with a focus on graph networks. Our recent findings suggest that (i) learning systems can approximate physical interaction at various granularities, ranging from rigid bodies to deformable shapes to fluids, (ii) the learned physical model implicitly encodes the physical object properties that govern the interaction, and (iii) incorporating physics explicitly into learning systems leads to improvement in both performance and data-efficiency for robot manipulation.
