ADAPTIVE PARALLEL DATA CLUSTERING WHEN LOADING A DATA STRUCTURE CONTAINING DATA CLUSTERED ALONG ONE OR MORE DIMENSIONS

摘要

Loading input data into a multi-dimensional clustering (MDC) table or other structure containing data clustered along one or more dimensions entails assembling blocks of data in a partial block cache in which each partial block is associated with a distinc t logical cell. A minimum threshold number of partial blocks may be maintained. Partial blocks may be spilled from the partial block cache to make room for new logical cells. Last partia l pages of spilled partial blocks may be stored in a partial page cache to limit I/O if the cel l associated with a spilled block is encountered later in the input data stream. Buffers may be reassign ed from the partial block cache to the partial page cache if the latter is filled. Parallelism m ay be employed for efficiency during sorting of input data subsets and during storage of blocks to secondary storage.