A Probabilistic Approach to the Semantic Interpretation of Building Facades
MetadataShow full item record
We present a probabilistic image-based approach to the semantic interpretation of building facades. We are motivated by the 4D Atlanta project at Georgia Tech, which aims to create a system that takes a collection of historical imagery of a city and infers a 3D model parameterized by time. Here it is necessary to recover, from historical imagery, metric and semantic information about buildings that might no longer exist or have undergone extensive change. Current approaches to automated 3D model reconstruction typically recover only geometry, and a systematic approach that allows hierarchical classification of structural elements is still largely missing. We extract metric and semantic information from images of facades, allowing us to decode the structural elements in them and their inter-relationships, thus providing access to highly structured descriptions of buildings. Our method is based on constructing a Bayesian generative model from stochastic context-free grammars that encode knowledge about facades. This model combines low-level segmentation and high-level hierarchical labelling so that the levels reinforce each other and produce a detailed hierarchical partition of the depicted facade into structural blocks. Markov chain Monte Carlo sampling is used to approximate the posterior over partitions given an image.