Dividing incoming data into multiple data streams and transforming the data for storage in a logical data object

摘要

Various embodiments for storing a logical object are provided. In one such embodiment, by way of example only, incoming data is divided corresponding to a logical data object into a plurality of independent streams, associating each data chunk of a plurality of obtained data chunks with a corresponding stream among the plurality of independent streams. At least one of the obtained data chunks and derivatives thereof is sequentially accommodated in accordance with an order the obtained chunks are received, while keeping the association with the corresponding streams. A global index is generated as a single meta-data stream accommodated in the logical data object and comprising information common to the plurality of independent streams and related to mapping between data in the logical data object and the obtained data chunks.

主权项

1. A method of storing a logical object in a computing environment by a processor device, comprising:
dividing incoming data corresponding to a logical data object into a plurality of independent streams; associating each data chunk of a plurality of obtained data chunks with a corresponding stream among the plurality of independent streams; transforming each data chunk on-the-fly to generate transformed data chunks, wherein transforming comprises at least one of compressing and encrypting each data chunk; sequentially accommodating the transformed data chunks and any derivatives thereof in accordance with an order the transformed chunks are received, while keeping the association with the corresponding streams; generating a global index as a single meta-data stream accommodated in the logical data object and comprising information common to each of the plurality of independent streams and related to mapping between data in the logical data object and the obtained data chunks, wherein the information comprises one of block allocation events, block de-allocation events, and a starting point of index streams for each independent stream; storing, in a transformed logical data object, indexed data in the global index as persistent data in a first portion of a data block; storing, in the transformed logical data object, non-indexed data in the global index as semi-persistent data in a second portion of the data block; and reusing free data blocks corresponding to different data streams using semi-persistent free-block management.