SYSTEMS AND MEHTODS FOR RELIABLY STORING DATA USING LIQUID DISTRIBUTED STORAGE

摘要

Embodiments provide methodologies for reliably storing data within a storage system using liquid distributed storage control. Such liquid distributed storage control operates to compress repair bandwidth utilized within a storage system for data repair processing to the point of operating in a liquid regime. Liquid distributed storage control logic of embodiments may employ a lazy repair policy, repair bandwidth control, a large erasure code, and/or a repair queue. Embodiments of liquid distributed storage control logic may additionally or alternatively implement a data organization adapted to allow the repair policy to avoid handling large objects, instead streaming data into the storage nodes at a very fine granularity.

主权项

1. A method for repair of source data comprising one or more source objects stored as multiple fragments distributed across multiple storage nodes of a storage system, wherein one or more fragments of the multiple fragments includes redundant data for the one or more source objects, the method comprising:
determining a portion of a source object of the one or more source objects to repair, wherein the portion of the source object to repair comprises data of at least one fragment of the multiple fragments; reading data of a plurality of fragments of the multiple fragments from a plurality of storage nodes of the multiple storage nodes; processing the data of the plurality of fragments read from the plurality of storage nodes to provide data of at least one additional fragment for the multiple fragments; writing the data of the at least one additional fragment to at least one storage node of the multiple storage nodes; and implementing liquid distributed storage control with respect to the reading data of the plurality of fragments and writing the data of the at least one additional fragment to control an average bandwidth for repair of source data within the storage system (R) as a function of data redundancy of the storage system provided by the one or more fragments including redundant data (β).