NON-VOLATILE BUFFERING FOR DEDUPLICATON

摘要

A system and method for storage of data is described where the data and commands received by a storage system is processed using at least a pair of redundant channels, configured so that received data buffered in a first channel is also buffered by a second channel prior to acknowledgement of the completion of the command execution. This permits a low latency of response to commands while securely storing the data. Data that is buffered in the first channel may be processed for storage, or for deduplication or compression prior to sending to the storage system subsequent to the acknowledgement of completion of the command and the data may then be purged from the data buffers in the redundant channels. A file identified as being smaller in size than the size allocated to associated metadata may be stored as part of the metadata without being sent to the storage system.

主权项

1. A data channel for a storage system, comprising:
a plurality of non-volatile buffers, each buffer communicating with an external communications interface, and at least two non-volatile buffers of the plurality of non-volatile buffers in mutual communication; a plurality of processors, each processor associated with a non-volatile buffer of the plurality of non-volatile buffers; wherein, (a) a first non-volatile buffer of the plurality of non-volatile buffers is configured to receive information comprising a command with a payload through the external communications interface, store the received information in the first non-volatile buffer and transmit the command and the payload to a second non-volatile buffer such that the received information and command is stored in the first and the second non-volatile buffers;
(b) a first processor associated with the first non-volatile buffer is configured to acknowledge an execution of the command associated with the received information to a user when the received information has been stored in the second-non-volatile buffer;(c) the first non-volatile buffer is configured to parse the received information and to associate information from received payloads with a common identifier so as to form a data block;(d) a first processor of the plurality of processors associated with the first non-volatile buffer is configured to dequeue the data block when (i) the data block has reached a predetermined size or (ii) processor resources are available to process the block; and;(e) the first processor configured to determine whether the data block has already been stored in a data storage system.