Consolidation for updated/deleted records in old fragments

摘要

A plurality of data records of a data set can be stored in a plurality of main part fragments, at least one of which is an old fragment stored on-disk. A number of one or more data records in the old fragment that have been marked for deletion can be determined to be greater than a threshold number, and the old fragment can be loaded into main system memory. A merge of the old fragment can be performed to remove the one or more data records marked for deletion.

主权项

1. A computer program product comprising a non-transitory machine-readable medium storing instructions that, when executed by at least one programmable processor, cause the at least one programmable processor to perform operations comprising:
storing, in a plurality of main part fragments, a plurality of data records that comprise a data set, the plurality of main part fragments comprising an old fragment stored on-disk; determining that a number of one or more data records in the old fragment that have been marked for deletion is greater than a threshold number; loading the old fragment into main system memory; performing a merge of the old fragment to remove the one or more data records marked for deletion; determining a total number of data records not marked for deletion in the old fragment plus a second old fragment that is adjacent to the old fragment that is less than a defined maximum fragment size; loading the second old fragment into main system memory; and performing a join of the old fragment and the second old fragment to generate a single old fragment.