Map-centric visual data association across seasons in a natural environment

Abstract

Vision is one of the primary sensory modalities of animals and robots, yet among robots it still has limited power in natural environments. Dynamic processes of Nature continuously change how an environment looks, which work against appearance-based methods for visual data association. As a robot is deployed again and again, the possibility of finding correspondences diminishes between surveys increasingly separated in time. This is a major limitation of intelligent systems targeted for precision agriculture, search and rescue, and environment monitoring. New approaches to data association may be necessary to overcome the variation in appearance of natural environments. This dissertation presents success with a map-centric approach, which builds on 3D vision to achieve visual data association across seasons. It first presents the new, Symphony Lake Dataset, which consists of fortnightly visual surveys of a 1.3 km lakeshore captured from an autonomous surface vehicle over three years. It then establishes dense correspondence as a technique to both provide robust visual data association and to eliminate the variation in viewpoint between surveys. Given a consistent map and localized poses, visual data association across seasons is achieved with the integration of map point priors and geometric constraints within the dense correspondence image alignment optimization. This algorithm is called Reprojection Flow. This dissertation presents the first work to see through the variation in appearance across seasons in a natural environment using map point priors and localized poses. The variation in appearance had a minimized effect on dense correspondence when anchored by accurate map points. Up to 37 surveys were transformed into year-long time-lapses at the scenes where their maps were consistent. This indicates that, at a time when frequent advancements are made towards robust visual data association, the spatial information in a map may be able to close the distance where hard cases have persisted between observations.