Yingjie Cao (The Hong Kong Polytechnic University), Xiaogang Zhu (The University of Adelaide), Dean Sullivan (University of New Hampshire), Haowei Yang (360 Security Technology Inc.), Lei Xue (Sun Yat-sen University), Xian Li (Swinburne University of Technology), Chenxiong Qian (University of Hong Kong), Minrui Yan (Swinburne University of Technology), Xiapu Luo (The Hong Kong Polytechnic University)
Double-fetch vulnerabilities arise when the kernel repeatedly retrieves data from user-space memory without ensuring consistency between the successive data fetches. This issue is especially severe in Real-Time Operating Systems (RTOS), where strict timing requirements limit the use of synchronization mechanisms like mutexes, thus favoring low-latency memory access at the cost of security. Most current detection techniques use static source code analysis, which cannot be applied to commercial off-the-shelf (COTS) RTOS with proprietary kernels. Dynamic methods that employ heuristic time-window thresholds to detect repeated cross-boundary memory accesses are used instead. However, these methods often produce a high number of false positives due to overly broad pattern recognition and lead to significant emulation overhead.
We introduce IsolatOS, a hardware-supported detection method that utilizes kernel isolation features to spot cross-boundary memory accesses that indicate double-fetch vulnerabilities. The main difficulty is in maintaining transparency while enforcing isolation boundaries without causing crashes in RTOS systems to boost efficiency. IsolatOS overcomes this by first implementing dynamic instrumentation that intercepts privileged accesses to user memory, recording metadata about accesses. Then exception recovery techniques upholds system stability during fault handling. At post-execution stage, the causal analysis examines violation traces to differentiate between legitimate dual accesses and exploitable double-fetches.
Evaluations across QNX, VxWorks, and seL4 demonstrate efficiency of IsolatOS, 70× runtime overhead reduction compared to emulation-based approach, identification of 42 unique vulnerabilities (39 vendor-confirmed, 2 CVEs assigned). These results validate hardware-assisted kernel isolation is a viable paradigm for double-fetch detection in COTS RTOS environments. We also demonstrate the real-world impact of our findings in automotive systems by exploiting them.