Multivariate Statistics and Machine Learning Under a Modern Optimization Lens

Abstract

Key problems of classification and regression can naturally be written as optimization problems. While continuous optimization approaches has had a significant impact in statistics, discrete optimization has played a very limited role, primarily based on the belief that mixed integer optimization models are computationally intractable. While such beliefs were accurate two decades ago, the field of discrete optimization has made very substantial progress.

Dr. Bertsimas will discuss how to apply modern first order optimization methods to find feasible solutions for classical problems in statistics, and mixed integer optimization to improve the solutions and to prove optimality by finding matching lower bounds.

Specifically, he will report results for the classical variable selection problem in regression currently solved by LASSO heuristically, least quantile regression, and factor analysis. He will also present an approach to build regression models based on mixed integer optimization. In all cases he will demonstrate that the solutions found by modern optimization methods outperform the classical approaches. Most importantly, he suggests that the belief widely held in statistics that mixed integer optimization is not practically relevant for statistics applications needs to be revisited.