System and method for evaluating malware detection rules

摘要

A malware detection rule is evaluated for effectiveness and accuracy. The detection rule defines criteria for distinguishing files having a characteristic of interest from other files lacking that characteristic, for instance, malicious files vs. benign files. The detection rule is applied to a set of unknown files. This produces a result set that contains files detected from among the set of unknown files as having the at least one characteristic of interest. Each file from the result set is compared to at least one file from a set of known files having the characteristic to produce a first measure of similarity, and to at least one file from a set of known files lacking the characteristic to produce a second measure of similarity. In response to the first measure of similarity exceeding a first similarity threshold, the detection rule is deemed effective. In response to the second measure of similarity exceeding a second similarity threshold, the detection rule is deemed inaccurate.

主权项

1. In a computing system that includes a processor, data storage, and input/output devices including a network interface device, and an operating system, a method for analyzing effectiveness and accuracy of a file detection rule, the method comprising:
obtaining, by the computing system, the detection rule, wherein the detection rule defines criteria for distinguishing files having at least one characteristic of interest from other files lacking the at least one characteristic of interest; applying, by the computing system, the detection rule to a set of unknown files, wherein prior to the applying of the detection rule, the at least one characteristic of interest is initially undetermined for each file of the set of unknown files;
wherein as a result of the applying of the detection rule, a result set is generated that contains files detected from among the set of unknown files as having the at least one characteristic of interest and excludes the other files lacking the at least one characteristic of interest; comparing, by the computing system, each file from the result set to at least one file from a first set of known files known to have the at least one characteristic of interest to produce a first measure of similarity, and to at least one file from a second set of known files known to lack the at least one characteristic of interest to produce a second measure of similarity;
wherein in response to the first measure of similarity exceeding a first similarity threshold, the detection rule is deemed effective; andwherein in response to the second measure of similarity exceeding a second similarity threshold, the detection rule is deemed inaccurate.