Zheng Zhang (UC RIverside), Haonan Li (UC Riverside), Xingyu Li (UC Riverside), Hang Zhang (Indiana University), Zhiyun Qian (University of California, Riverside)
Bug bisection has been an important security task that aims to understand the ranges of software versions impacted by the bug, i.e., identifying the commit that introduced the bug. However, traditional patch-based bisection methods are faced with several significant barriers: For example, they assume that the bug-inducing commit (BIC) and the patch commit modify the same functions, which is not always true; they often rely purely on code changes, while the commit message frequently contains a wealth of vulnerability-related information; they are also based on simple heuristics (e.g., assuming the BIC initializes lines deleted in the patch) and lack a logical analysis of the vulnerability.
In this paper, we make the observation that Large Language Models (LLMs) are well positioned to break the barriers of existing solutions, e.g., comprehend both textual data and code well in patches and commits. We develop a comprehensive multi-stage pipeline leveraging LLMs to (1) take advantage of full patch information, (2) have LLM assess logic of the bug and the likelihood of a commit being the one that introduced the bug, and (3) gradually narrow down the candidate with multiple down-select processes. In our evaluation, we demonstrate that our approach achieves significantly better accuracy than the state-of-the-art solution by more than 38%. Our results further confirm that the comprehensive multi-stage pipeline is essential, as it improves accuracy by 60% over naive LLM application.