Learning Locomotion: From Simulation to Real World

Abstract

Deep Reinforcement Learning (DRL) holds the promise of designing complex robotic controllers automatically. In this talk, I will discuss two different approaches to apply deep reinforcement learning to learn locomotion controllers for legged robots. The first approach is through sim-to-real transfer. Due to safety concerns and limited data, most of the training is conducted in simulation. However, controllers learned in simulation usually perform poorly on real robots. I will present a set of techniques to overcome this sim-to-real gap. The second approach is to directly learn in the real world. Due to the complexity and diversity of the real environments, building a simulation that can faithfully model the real world is not always feasible. Having the ability to learn on the fly and adapt quickly in real-world scenarios is crucial for large-scale deployment of robots. I will discuss the challenges of training legged robots in the real world and various ways to address these challenges.