Compiler Techniques for Stealthy Protections

01 November 2020 → 31 October 2024
Research Foundation - Flanders (FWO)
Research disciplines
  • Engineering and technology
    • Other computer engineering, information technology and mathematical engineering not elsewhere classified
Software Security and Protection Code Transformations Compiler Technology
Project description

In software attacks, attackers need to identify the attack vectors. When injected software protections, such as protections against man-at-the-end attacks (i.e., unauthorized reverse engineering and tampering) and security fixes of vulnerabilities are easily located because they lack stealth, this significantly eases the attacks. Stealth has long been recognized as an important aspect of software security. Little research has been done, however, to make the integration of protections or fixes stealthier. I will study a holistic approach to integrate new security functionality into software in a much stealthier way. At several levels of abstraction (data structure APIs, compiler IR, assembly code), I aim to transform new functionality when it is compiled and linked into existing software such that the new code becomes as indistinguishable as possible from the existing code in light of common attack methods to identify code, such as pattern matching, taint tracking, statistical analysis, and identification of invariants. To reach my goal, the transformations that I plan to research aim for code similarity as well as for code reuse. My plan includes the whole compiler tool flow, including source-to-source rewriting, IR transformations in the compiler middle end, adapted code generation in the back end, and binary rewriting. I will gradually study and evaluate more complex approaches, ranging from best-effort, over iterative feedback-guided, to machine learning based.