Modeling Rich Structured Data via Kernel Distribution Embeddings
MetadataShow full item record
Real world applications often produce a large volume of highly uncertain and complex data. Many of them have rich microscopic structures where each variable can take values on manifolds (e.g., camera rotations), combinatorial objects (e.g., texts, graphs of drug compounds) or high dimensional continuous domains (e.g., images and videos). Furthermore, these problems may possess additional macroscopic structures where the large collections of observed and hidden variables are connected by networks of conditional independence relations (e.g., in predicting depth from still images, and forecasting in time-series). Most previous learning algorithms for problems with such rich structures rely heavily on linear relations and parametric models where data are typically assumed to be multivariate Gaussian or discrete with a relatively small number of values. Conclusions inferred under these restricted assumptions can be misleading, if the underlying data generating processes contain nonlinear, non-discrete, or non-Gaussian components. How can we find a suitable representation for nonlinear and non-Gaussian relationships in a data-driven fashion? How can we exploit conditional independence structures between variables in rich structured setting? How can we design efficient algorithms to solve challenging nonparametric problems involving large amount of data? In this talk, I will introduce a nonparametric representation for distributions called kernel embeddings to address these questions. The key idea of the method is to map distributions to their expected features (potentially infinite dimensional), and given evidence, update these new representations solely in the feature space. Compared to existing nonparametric representations which are largely restricted to vectorial data and usually lead to intractable algorithms, very often kernel distribution embeddings lead to simpler, faster and more accurate algorithms in a diverse range of problems such as organizing photo albums, understanding social networks, retrieving documents across languages, predicting depth from still images and forecasting sensor time-series.