Method and system for transformation of logical data objects for storage

摘要

Systems capable of transformation of logical data objects for storage and methods of operating thereof are provided. One method includes identifying among a plurality of requests addressed to the storage device two or more “write” requests addressed to the same logical data object in a distributed network, deriving data chunks corresponding to identified “write” requests and transforming the derived data chunks, grouping the transformed data chunks in accordance with a predefined criteria, generating a grouped “write” request to the storage device, and providing mapping in a manner facilitating one-to-one relationship between the data in the obtained data chunks and the data to be read from the transformed logical object. The method further includes obtaining an acknowledging response from the storage device, multiplying the obtained acknowledging response, and sending respective acknowledgements to each source that initiated each respective “write” request.

主权项

1. A method of operating a transformation system operatively coupled to a plurality of servers and to a plurality of storage devices via a distributed network and operable to write a plurality of data chunks to a plurality of transformed logical data objects (LOs) stored in the storage device, each given transformed logical data object corresponds to a respective original logical data object and configured to sequentially accommodate a plurality of variable size data chunks corresponding to the original LO, transformed by the transformation system and accommodated in accordance with an order in which said chunks are received; the method comprising:
transmitting, by the plurality of servers, a plurality of data chunks and a plurality of “write” requests to write the plurality of data chunks to the plurality of storage devices, each data chunk associated with a respective “write” request; identifying among a plurality of “write” requests each “write” request addressed to the same transformed LO; deriving data chunks corresponding to each identified “write” request and processing the derived data chunks, thereby giving rise to transformed data chunks; grouping the transformed data chunks that were transmitted by the same server in a respective group, thus giving rise to a group of transformed data chunks; determining if a respective transformed LO for accommodating a particular group of transformed data chunks in a respective storage device includes sufficient free space to accommodate the particular group of transformed data chunks; if the respective transformed LO includes sufficient free space to accommodate the particular group of transformed data chunks:
generating a grouped “write” request to the respective transformed LO, andwriting the particular group of transformed data chunks to the respective transformed LO; if the respective transformed LO includes insufficient free space to accommodate the particular group of transformed data chunks:
determining if the particular group of transformed data chunks meets a predefined criteria related to a size of the particular group of transformed data chunks,if the size of the particular group of transformed data chunks meets the predefined criteria:
allocating a new transformed LO in the respective storage device,splitting the grouped “write” request into a plurality of parts,generating a first grouped “write” request to the respective transformed LO, the first grouped “write” request comprising a first part of the plurality of parts, andgenerating a second grouped “write” request to the new transformed LO, the second grouped “write” request comprising a second part of the plurality of parts,writing the first part of the plurality of parts to the respective transformed LO, andwriting the second part of the plurality of parts to the new transformed LO, andif the size of the particular group of transformed data chunks fails to meet the predefined criteria:
allocating the new transformed LO, andwriting the particular group of transformed data chunks to the new transformed LO; and mapping between the data in the original LO and the data accommodated in the respective transformed LO in a manner facilitating one-to-one relationship between the data in the transformed data chunks and the data to be read from the corresponding one of the respective transformed LO, the respective transformed LO and new transformed LO, and the new transformed LO.