Hardware Supported Anomaly Detection: down to the Control Flow Level

Abstract

Modern computer systems are plagued with security flaws, making them vulnerable to various malicious attacks. Intrusion detection systems have been proposed to protect computer systems from unauthorized penetration. Detecting an attack early on pays off since further damage is avoided and resilient recovery could be adopted. An intrusion detection system monitors dynamic program behavior against normal program behavior and raises an alert when anomaly is detected. The normal behaviour is learnt by the system through training and profiling. However, all current intrusion detection systems are purely software based and thus suffer from huge performance degradation due to constant monitoring operations inserted in the application code. Due to the potential performance overhead, software based solutions cannot monitor the program behavior at a very fine level of granularity, thus leaving potential security holes as shown in [5]. In this paper, we propose a hardware-based approach to verify the control flow of target applications dynamically and to detect anomalous executions. With hardware support, our approach offers multiple advantages over software based solutions including near zero performance degradation, much stronger detection capability (a larger variety of attacks get detected) and zero-latency reaction upon anomaly and thus much better security.