Concolic Execution Tailored for Hybrid Fuzzing

Abstract

Recently, hybrid fuzzing, which combines fuzzing and concolic execution, has been highlighted to overcome limitations of both techniques. Despite its success in contrived programs such as DARPA Cyber Grand Challenge (CGC), it still falls short in finding bugs in real-world software due to its low performance of existing concolic executors.

To address this issue, this dissertation suggests and demonstrates concolic execution tailored for hybrid fuzzing with two systems: QSYM and Hybridra. First, we present QSYM, a binary-only concolic executor tailored for hybrid fuzzing. It significantly improves the performance of conventional concolic executors by removing redundant symbolic emulations for a binary. Moreover, to efficiently produce test cases for fuzzing, even sacrificing its soundness, QSYM introduces two key techniques: optimistic solving and basic block pruning. As a result, QSYM outperforms state-of-the-art fuzzers, and, more importantly, it found 13 new bugs in eight real-world programs, including file, ffmpeg, and OpenJPEG.

Enhancing the key idea of QSYM, we discuss Hybridra, a new concolic executor for file systems. To apply hybrid fuzzing for file systems, which are gigantic and convoluted, Hybridra employs compilation-based concolic execution to boost concolic execution leveraging the existing of source code. Moreover, Hybridra introduces a new technique called staged reduction, which combines existing heuristics to efficiently generate test cases for file systems. Consequently, Hybridra outperforms a state-of-the-art file system fuzzer, Hydra, by achieving higher code coverage, and successfully discovered four new bugs in btrfs, which has been heavily tested by other fuzzers.