Main-memory database checkpointing

摘要

The present invention extends to methods, systems, and computer program products for main-memory database checkpointing. Embodiments of the invention use a transaction log as an interface between online threads and a checkpoint subsystem. Using the transaction log as an interface reduces synchronization overhead between threads and the checkpoint subsystem. Transactions can be assigned to files and storage space can be reserved in a lock free manner to reduce overhead of checkpointing online transactions. Meta-data independent data files and delta files can be collapsed and merged to reduce storage overhead. Checkpoints can be updated incrementally such that changes made since the last checkpoint (and not all data) are flushed to disk. Checkpoint I/O is sequential, helping ensure higher performance of physical I/O layers. During recovery checkpoint files can be loaded into memory in parallel for multiple devices.

主权项

1. At a computer system, the computer system including one or more processors, system memory, and durable storage, the computer system maintaining an in-memory database in system memory, a method for updating a checkpoint for the in-memory database, the method comprising:
committing a transaction, the results of the transaction modifying the content of the in-memory database, the transaction having a timestamp, the timestamp indicating an associated time the transaction was committed relative to other transactions; generating checkpoint data for the transaction from the results of the transaction, the checkpoint data including versions of one or more inserted portions of data inserted into the in-memory database and including identifiers for one or more deleted portions of data deleted from the in-memory database; appending the checkpoint data to a checkpoint, the checkpoint including a directory file referencing one or more data files and one or more corresponding delta files that makeup the checkpoint, including:
determining that the timestamp is within a specified timestamp range for a referenced data file, the referenced data file configured to store any inserted portions of data inserted into the in-memory database within the specified timestamp range;appending the one or more inserted portions of data to the referenced data file;for each of the one or more deleted portions of data:
identifying a corresponding insert operation that inserted the deleted portion of data into the in-memory database;matching the deleted portion of data to the corresponding insert operation by locating a timestamp for a transaction that included the corresponding insert operation;determining that the located timestamp is within a second specified time range for another referenced data file;appending the identifier for the deleted portion of data to a delta file corresponding to the other data file, the delta file configured to store identifiers for any deleted portions of data deleted from the in-memory database during the second specified time range.