Hardware-enabled prevention of code reuse attacks

摘要

Described systems and methods allow protecting a host computer system from malware, such as return-oriented programming (ROP) and jump-oriented programming (JOP) exploits. In some embodiments, a processor of the host system is endowed with two counters configured to store a count of branch instructions and a count of inter-branch instructions, respectively, occurring within a stream of instructions fetched by the processor for execution. Exemplary counted branch instructions include indirect JMP, indirect CALL, and RET on x86 platforms, while inter-branch instructions consist of instructions executed between two consecutive counted branch instructions. The processor may be further configured to generate a processor event, such as an exception, when a value stored in a counter exceeds a predetermined threshold. Such events may be used as triggers for launching a malware analysis to determine whether the host system is subject to a code reuse attack.

主权项

1. A host system comprising a hardware processor, the hardware processor including:
a branch counter register configured to store a count of branch instructions occurring within a sequence of instructions executed by the hardware processor; an inter-branch instruction counter register configured to store a count of instructions occurring between two consecutive branch instructions within the sequence of instructions; and a counter control unit connected to the branch counter register, to the inter-branch instruction counter register, and to an instruction decoder module of the hardware processor, wherein the counter control unit comprises hardware logic configured to:
perform branch monitoring, andtrigger a switch event within the hardware processor according to a value stored in the branch counter register and according to a value stored in the inter-branch instruction counter register, wherein the switch event causes the hardware processor to switch from executing the sequence of instructions to executing an event handler routine, wherein branch monitoring comprises:
determining whether a selected instruction of the sequence of instructions is a branch instruction;in response to determining whether the selected instruction is a branch instruction, when the selected instruction is a branch instruction, incrementing the value stored in the branch counter register; andin response to determining whether the selected instruction is a branch instruction, when the selected instruction is not a branch instruction, incrementing the value stored in the inter-branch instruction counter register; and wherein the hardware processor is further configured, in response to the switch event, to:
identify a target execution thread according to the sequence of instructions; andin response to identifying the target execution thread, determine whether the target execution thread is performing a code-reuse attack.