Image retrieval and geolocalization with deep learning

Abstract

This work studies image localization task and explores image ranking/retrieval approach. Deep Learning has advanced many computer vision task including image retrieval; in addition, location tagged image data has become increasingly abundant. The first contribution is a study of image geolocalization at planet scale (Im2GPS: predicting GPS coordinate from image data) comparing 2 deep learning approaches: image classification and image retrieval. We analyze the trade off between localization accuracy at different granularity levels. Image retrieval approach has great advantage when it comes to geolocalization at fine levels (street, city) and still competitive at coarse levels (country, continent). Next, we investigate different architectures for matching and retrieving crossview images. The application is to do localization using image retrieval approach where the query images are normal streetview images, but reference images in the database are overhead viewpoint (satellite images). The third contribution is exploring state of the art Deep Metric Learning (DML) techniques in image retrieval. We first look at it in the context of fine grained image retrieval, which is much well studied in the literature, and analyze generalization performance when switching embedding layer. Lastly, we apply DML techniques to training deep networks for image retrieval and Im2GPS geolocalization task. Our experiment shows that DML trained systems outperform a classification trained system as feature extractors, result in better image retrieval and geolocalization performance.