Method and system for transformation of logical data objects for storage

摘要

A method and system for transforming a logical data object for storage in a storage device operable with at least one storage protocol, creating, reading, writing, optimization and restoring thereof. Transforming the logical data object comprises creating in the storage device a transformed logical data object comprising a header and one or more allocated accommodation sections with predefined size; transforming one or more sequentially obtained chunks of obtained data corresponding to the transforming logical data object thus giving rise to the transformed data chunks; and sequentially accommodating the processed data chunks into said accommodation sections in accordance with an order said chunks received, wherein said accommodation sections serve as atomic elements of transformation/de-transformation operations during input/output transactions on the logical data object. The processing may comprise two or more data transformation techniques coordinated in time, concurrently executing autonomous sets of instructions, and provided in a manner facilitating preserving the sequence of processing and accommodating the processed data chunks.

主权项

1. A method of transforming a logical data object for storage in a long-term storage device using at least one storage network protocol, said method comprising:
in response to a respective request, creating in the long-term storage device a transformed logical data object comprising a header and a plurality of accommodation sections with a predefined size; receiving, via a storage network, a plurality of data chunks comprising a variable size and that correspond to the transformed logical data object at a transformation system; comparing a characteristic of the plurality of data chunks to a predefined criterion related to an actual time of compression or an estimated time of compression, the estimated time of compression including an estimated time necessary to compress data that, when compressed, would substantially amount to a compressed section size; compressing, by the transformation system, the plurality of data chunks thus giving rise to a plurality of compressed data chunks, wherein each of said plurality of compressed data chunks comprises compressed data resulting from said compressing and at least two of the compressed data chunks comprise different sizes; accumulating the compressed data chunks to form the plurality of accommodation sections in the long-term storage device if the predefined criterion matches the characteristic of the received plurality of data chunks, the compressed data chunks accumulated in each accommodation section until an accommodation section includes an amount of compressed data chunks matching the predefined size; identifying each of the plurality of data chucks as being non-compressed data or partially compressed data and accumulating the non-compressed data or the partially compressed data in one of the plurality of accommodation sections if the predefined criterion does not match the characteristic, wherein the identification of the received plurality of data chunks as being non-compressed data or partially compressed data is included as entries in an index; sequentially accommodating the plurality of compressed data chunks into said plurality of accommodation sections in accordance with an order said plurality of data chunks are received; repeating the comparing and at least one of the compressing step and the identifying step for subsequently received plurality of data chunks, and accumulating subsequently compressed data chunks and non-compressed data chunks or subsequently compressed data chunks and partially compressed data chunks to form an accommodation section having a size matching the predefined size, and mapping the data in the logical data object and the data accommodated in the plurality of accommodation sections, wherein:
the predefined size of the plurality of accommodation sections are equal to each other, andthe non-compressed data chunks and the partially compressed data chunks are data chunks in which compression was interrupted.