Throughput-sensitive redundancy encoding schemes for data storage

摘要

Techniques for optimizing data storage are disclosed herein. In particular, methods and systems for implementing redundancy encoding schemes with data storage systems are described. The redundancy encoding schemes may be scheduled according to system and data characteristics. The schemes may span multiple tiers or layers of a storage system. The schemes may be generated, for example, in accordance with a transaction rate requirement, a data durability requirement or in the context of the age of the stored data. The schemes may be designed to rectify entropy-related effects upon data storage. The schemes may include one or more erasure codes or erasure coding schemes. Additionally, methods and systems for improving and/or accounting for failure correlation of various components of the storage system, including that of storage devices such as hard disk drives, are described.

主权项

1. A computer-implemented method for optimizing data storage, comprising:
receiving first information about one or more access-related characteristics with respect to data stored on a storage system; determining, based at least in part on the received first information, a redundancy encoding scheme to be applied to the data; receiving second information about the one or more access-related characteristics, the second information being distinct from the first information; modifying the redundancy encoding scheme based at least in part on the received second information; and applying the modified redundancy encoding scheme to at least a portion of the data.