METHOD FOR MASSIVELY PARALLEL MULTI-CORE TEXT INDEXING

摘要

There is provided, in a parallel pipelined structure on a multi-core device, a method for parallel pipelined multi-core indexing. The method includes generating one or more single document indexes respectively corresponding to one or more single documents of a given data stream. The method further includes generating one or more multi-document interval-based hash tables from the one or more single document indexes. The method also includes generating a global hash table formed from merging one or more of the multi-document interval-based hash tables, the global hash table representing a collective index for all of the single documents for which the one or more single document indexes were generated.