Method and system for data compression at a storage system

摘要

A computer-implemented method for compressing data is disclosed. The method starts with determining a way to read a received data block in its native endian format of at a storage system, where the data block contains a set of data and the determination is based on sampling a subset from a set of data and checking variation of the values. The method selects a base value for the data block based on the determined way to read the data block and generates a set of updated data, where each value of the set of updated data corresponds to the base value and an original value. The method separates each data within the set of updated data into two portions with different bit-value distribution patterns and compresses one portion with a first algorithm while compresses another portion with a second algorithm different from the first.

主权项

1. A computer-implemented method comprising:
determining a way to read a received data block in its native endian format at a storage system, by sampling a subset from a set of data contained in the data block and checking variation of values of the subset of data; separating each data contained in the data block into a first portion of data having a first bit-value distribution pattern and a second portion of data having a second bit-value distribution pattern based on the determined way to read the data block; and compressing the first portion using a first compression algorithm and compressing the second portion using a second compression algorithm that is different from the first compression algorithm.