Dynamic robust sparse modeling and sampling of high-dimensional data streams for online monitoring and change detection

Abstract

This thesis contributes to the area of System Informatics and Control (SIAC) to develop systematic and dynamic methodologies for effective monitoring and change detection in complex systems. We propose sampling method named “Correlation based Dynamic Sampling” (CDS). It leverages spatial dependencies within data streams to improve decision making, when deploying sensors with resource limitations. Furthermore, we develop a dimension reduction method named “Robust Sparse Principal Component Analysis” (RS-PCA), that is designed to robustly estimate a lower dimensional subspace by exploiting sparse data structures. The probabilistic approach for modelling offers a direct medium for making inferences on system conditions. Additionally, we extends the aforementioned RS-PCA procedure for implementation in dynamic systems. The proposed adaptive RS-PCA method reduces the false alarm rate that may result from implementing static procedures. These strategies are implemented on several exemplary systems to assess their capability for real time application in practical scenarios.