COMPRESSED DISTRIBUTED STORAGE SYSTEMS AND METHODS FOR PROVIDING SAME

摘要

Disclosed are embodiments of a compressed distributed storage system that is designed to satisfy: reliability; minimum storage; efficient update; cost-effective access. An exemplary system can comprise a splitter, an encoder, a parameterizer, and a compressor. In contrast to the prior art, the encoding is performed before the compression. Furthermore, in the exemplary system parameterization, data classification, and memory-assisted compression are key features in efficient compression. The splitter can split an input data file into a plurality of original segments. The encoder can perform fault-tolerant encoding on the plurality of original segments, providing plurality of redundant segments. The parameterizer can classify each redundant segment and form and memorize statistics (context) of each class of the redundant segments. With the class-based context, each redundant segment can be compressed and later decompressed individually. Each compressed redundant segment can be stored at a storage unit of a distributed storage system.