FlexFilt: Towards Flexible Instruction Filtering for Security

As the complexity of software applications increases, there has been a growing demand for intra-process memory isolation. The commercially available intra-process memory isolation mechanisms in modern processors, e.g., Intel’s memory protection keys, trade-off between efficiency and security guarantees. Recently, researchers have tended to leverage the features with low security guarantees for intra-process memory isolation. Subsequently, they have relied on binary scanning and runtime binary rewriting to prevent the execution of unsafe instructions, which improves the security guarantees. Such intra-process memory isolation mechanisms are not the only security solutions that have to prevent the execution of unsafe instructions in untrusted parts of the code. In fact, we identify a similar requirement in a variety of other security solutions. Although binary scanning and runtime binary rewriting approaches can be leveraged to address this requirement, it is challenging to efficiently implement these approaches.