METHOD AND SYSTEM FOR IMPLEMENTING EFFICIENT CLASSIFICATION AND EXPLORATION OF DATA

摘要

Disclosed is a system, method, and computer program product for analyzing sets of data in an efficient manner, such that analytics can be effectively performed over that data. Classification operations can be performed to generate groups of similar log records. This permits classification of the log records in a cohesive and informative manner.

主权项

1. A method comprising:
receiving a plurality of log records from a processing system, the plurality of log records comprising one or more first log records and one or more second log records; comparing the one or more first log records to the one or more second log records to determine how similar the one or more first log records are to the one or more second log records, the one or more first log records compared to the one or more second log records by independently tokenizing the one or more first log records into a first plurality of tokens and the one or more second log records into a second plurality of tokens, where a similarity value is generated that corresponds to a degree of overlap, in terms of both token content and position, between the first plurality of tokens and the second plurality of tokens; and classifying the one or more first log records and the one or more second log records into either same or different groups based at least in part on whether the similarity value is less than a threshold level.