Reinforcement learning based active localization for precise manipulation

Abstract

Precision during positioning of a robot arm is inherently limited by the sensing capabilities of the robot. In the case of fully on-board sensing, this limitation is further exacerbated by inconsistencies in the expected and actual structure of the robot environment. Further, precision is also affected by uncertainty in the estimate of the robot pose. While these sources of errors can never be fully eliminated, we can at least devise an optimal measurement policy for a given sensor configuration and robot environment. That is exactly what we try to do in this thesis. Such a policy would ensure that measurements are taken from places which lead to an optimal increase the localisation precision. For our purpose, we assume that the robot end-effector is equipped with a sensor setup consisting of an array of highly accurate 1D laser rangefinders (pointlasers). Pointlasers provide a cheaper and more accurate alternative to a 3D LiDAR, at the expense of measurements which are sparsely distributed in the rotation space. We treat this policy estimation problem as an active localization problem and set up a 3D simulation environment consisting of the CAD model of a building. Inside this environment, we simulate different orientations of the pointlaser array and train a reinforcement learning (RL) agent which predicts the best orientation for a given position in the CAD model. Using reduction in uncertainty (information gain) as the reward, we come up with a policy which takes reliable measurements and thus, should lead to precise positioning of the end-effector. We train the agent using an on-policy RL algorithm and present the results on a number of test CAD models.