Learning from the Field: Physically-based Deep Learning to Advance Robot Vision in Natural Environments

Abstract

Field robotics refers to the deployment of robots and autonomous systems in unstructured or dynamic environments across air, land, sea, and space. Robust sensing and perception can enable these systems to perform tasks such as long-term environmental monitoring, mapping of unexplored terrain, and safe operation in remote or hazardous environments. In recent years, deep learning has led to impressive advances in robotic perception. However, state-of-the-art methods still rely on gathering large datasets with hand-annotated labels for network training. For many applications across field robotics, dynamic environmental conditions or operational challenges hinder efforts to collect and manually label large training sets that are representative of all possible environmental conditions a robot might encounter. This limits the performance and generalizability of existing learning-based approaches for robot vision in field applications. In this talk, I will discuss my work to develop approaches for unsupervised learning to advance perceptual capabilities of robots in underwater environments. The underwater domain presents unique environmental conditions to robotic systems that exacerbate the challenges in perception for field robotics. To address these challenges, I leverage physics-based models and cross-disciplinary knowledge about the physical environment and the data collection process to provide constraints that relax the need for ground truth labels. This leads to a hybrid model-based, data-driven solution. I will also present work that relates this framework to challenges for autonomous vehicles in other domains.