Object-based Visual SLAM: How Object Identity Informs Geometry

Abstract

Objects are rich information sources about the environment. A 3D model of the objects, together with their semantic labels, can be used for camera localization as well as for cognitive reasoning about the environment. However, traditional frameworks for scene reconstruction usually map a cloud of points using structure-from-motion techniques, but do not provide objects representation. On the other side, robotics object-based mapping mainly focus on adding cognitive representations to a metric or topologic map built using traditional SLAM techniques. In this work we propose a framework for environment modeling by representing the objects in the scene, detected by an object recognition and segmentation technique. The key idea is to incorporate the resulting image segments and labels into a global inference engine in order to build simple geometric models for the objects. For now, we consider the perfect object recognition case, where we know the exact object identities, testing our approach using coarsely hand-annotated images captured by a robot carrying an omnidirectional camera. We found that the resultant object locations and sizes are fully compatible with what is expected, and the inferred robot trajectory is improved when compared to that recovered using odometry only.