Log structured content addressable deduplicating storage

摘要

A log structured content addressable deduplicated data storage system may be used to store deduplicated data. Data to be stored is partitioned into data segments. Each unique data segment is associated with a label. The storage system maintains a transaction log. Mutating storage operations are initiated by storing transaction records in the transaction log. Additional transaction records are stored in the log when storage operations are completed. Upon restarting an embodiment of the data storage system, the transaction records from the transaction logs are replayed to recreate the state of the data storage system. The data storage system updates file system metadata with transaction information while a storage operation associated with the file is being processed. This transaction information serves as atomically updated transaction commit points, allowing fully internally consistent snapshots of deduplicated volumes to be taken at any time.

主权项

1. A method of modifying data in a data storage system, the method comprising:
identifying a storage label and storage data associated with a storage operation; generating a first transaction record indicating an initiation of the storage operation, wherein the first transaction record includes a transaction identifier; storing the first transaction record including the transaction identifier in a transaction log data structure; modifying a portion of the data storage system associated with the storage label and the storage data based on the storage operation; identifying a file system file associated with the portion of the data storage system; performing an atomic file system operation on the file system file to store the transaction identifier in file system metadata, wherein the file system metadata is separate from the transaction log data structure; determining that the storage operation was successfully committed if (1) the first transaction record exists in the transaction log data structure, but a second transaction record indicating that the storage operation was completed does not exist in the transaction log data structure, (2) the first transaction record includes the transaction identifier, and (3) the file system metadata also includes the transaction identifier; and determining that the storage operation was unsuccessfully committed if (1) the first transaction record exists in a transaction log data structure, but the second transaction record indicating that the storage system operation was completed does not exist in the transaction log data structure, (2) the first transaction record includes the transaction identifier, and (3) the file system metadata does not include the transaction identifier.