Methods and apparatus for detecting and displaying similarities in large data sets

摘要

Interactive Methods and apparatus for studying similarities of values in very large data sets. The methods and apparatus employ a dotplot in an interactive graphical user interface to make the relationship between the similarities and the data set visible. A variety of filtering, weighting, and compression techniques make it possible to employ the dot plot with sequences of more than 10,000 tokens and to interactively magnify the dot plot, change weighting and display quantization, and view the underlying data. Also disclosed is a technique which is employed in the apparatus for identifying long sequences of similar tokens. The apparatus is used in the study of large bodies of text and code.