Olivia Lucca Fraser | The Evolutionary Exploration of Emergent Execution: Genetic Programming and Weird Machines | December 7, 2021

Olivia Lucca Fraser - Special Circumstances (research consultancy)

Abstract: The process of exploiting or "hacking" a software vulnerability can, in many cases, be understood as the process of discovering and then programming what Halvar Flake has called a "weird machine" -- a spontaneous virtual machine that supervenes on the intended finite state machine that the vulnerable software in question implements. A weird machine has its own peculiar instruction set and program semantics, designed by no one and existing entirely by accident. In this seminar, I will be demonstrating the utility of genetic programming (GP) as a technique for exploring the space of programs implicit in a particular variety of "weird machine": the variety exploited by return-oriented programming (ROP), a remote code execution technique that, over the past two decades, has been used by attackers to subvert the separation of writeable and executable memory imposed by various operating systems as a security feature. I will show how it is possible to evolve ROP payloads and breed them to carry out various tasks, by means of applying certain selective pressures to "populations" of integer sequences, equipped with the genetic operators of mutation and crossover. We will look particularly closely at an observed correlation between the availability of crossover (a crude form of sexual reproduction) and the likelihood of a population to discover recombinable ROP "gadgets" when initialized with a pool of random integers.