Systems and methods for feedback driven regression testing

摘要

Systems and methods for automatically testing one or more versions of a compiler of are disclosed. A compiler is instrumented to generated data exposing various internal decisions and/or actions made by the compiler. Subsequently, multiple distinct versions of the compiler are executed to compile a code corpus associated with a particular programming language. Output (including instrumentation output) from the compilation of the code corpus for each version of the compiler is obtained and compared to identify behavioral changes that may exist between the various versions of the compiler.

主权项

1. A method for testing a compiler comprising:
compiling a code corpus comprising a plurality of test programs corresponding to a particular programming language, the code corpus being compiled by a first version of a compiler to generate first compiler output; compiling the code corpus using a second version of the compiler to generate second compiler output, the second version of the compiler including different functionality than the first version of the compiler; retrieving, by one or more computing devices, at least one test corresponding to the first compiler output and the second compiler output; and executing, by the one or more computing devices, the at least one test to generate a regression result for the second version of the compiler.