Robust approaches and optimization for 3D data

Abstract

We introduce a robust, purely geometric, representation framework for fundamental association and analysis problems involving multiple views and scenes. The framework utilizes surface patches / segments as the underlying data unit, and is capable of effectively harnessing macro scale 3D geometry in real world scenes. We demonstrate how this results in discriminative characterizations that are robust to high noise, local ambiguities, sharp viewpoint changes, occlusions, partially overlapping content and related challenges. We present a novel approach to find localized geometric associations between two vastly varying views of a scene, through semi-dense patch correspondences, and align them. We then present means to evaluate structural content similarity between two scenes, and to ascertain their potential association. We show how this can be utilized to obtain geometrically diverse data frame retrievals, and resultant rich, atemporal reconstructions. The presented solutions are applicable over both depth images and point cloud data. They are able to perform in settings that are significantly less restrictive than ones under which existing methods operate. In our experiments, the approaches outperformed pure 3D methods in literature. Under high variability, the approaches also compared well with solutions based on RGB and RGB-D. We then introduce a robust loss function that is generally applicable to estimation and learning problems. The loss, which is nonconvex as well as nonsmooth, is shown to have a desirable combination of theoretical properties well suited for estimation (or fitting) and outlier suppression (or rejection). In conjunction, we also present a methodology for effective optimization of a broad class of nonsmooth, nonconvex objectives --- some of which would prove problematic for popular methods in literature. Promising results were obtained from our empirical analysis on 3D data. Finally, we discuss a nonparametric approach for robust mode seeking. It is based on mean shift, but does not assume homoscedastic or isotropic bandwidths. It is useful for finding modes and clustering in irregular data spaces.