A Feature-Oriented Corpus for Understanding, Evaluating and Improving Fuzz Testing

Fuzzing is a promising technique for detecting security vulnerabilities. Newly developed fuzzers are typically evaluated in terms of the number of bugs found on vulnerable programs/binaries. However, existing corpora usually do not capture the features that prevent fuzzers from finding bugs, leading to ambiguous conclusions on the pros and cons of the fuzzers evaluated. In this paper, we propose to address the above problem by generating corpora based on search-hampering features. As a proof-of-concept, we designed FEData, a prototype corpus that currently focuses on three search-hampering features to generate vulnerable programs for fuzz testing. Unlike existing corpora that can only answer "how", FEData can also further answer "why" by exposing (or understanding) the reasons for the identified weaknesses in a fuzzer. The "why" information serves as the key to the improvement of fuzzers. Based on the "why" information, our FEData programs enabled us to identify the weakness of AFLFast, called cycle explosion, behind. We further developed an improved version of AFLFast, called AFLFast+, which has overcome the cycle explosion problem. AFLFast+ retains the efficiency of AFLFast in path search while maintaining or even surpassing the bug-finding capability of AFL for the corpus evaluated.