Byte-aligned dictionary-based compression and decompression

摘要

A system reads a metadata byte in a compressed data fragment. The metadata byte includes information pertaining to a sequence of instruction sets to decode data in the compressed data fragment. The sequence of instruction sets follows the metadata byte and one or more literal data sets corresponding to the sequence of instruction sets follows the sequence of instruction sets. The system determines a location of the one or more literal data sets in the compressed data fragment using the metadata byte prior to reading the sequence of instruction sets. The system determines the lengths of the one or more literal data sets based on the sequence of instruction sets and copies the one or more literal data sets, as a batch job for the sequence of instruction sets, to an in-memory data store based on the location and using the lengths.

主权项

1. A method comprising:
reading, by a computing device, a metadata byte in a compressed data fragment, wherein the metadata byte comprises information pertaining to a sequence of instruction sets to decode data in the compressed data fragment, the sequence of instruction sets following the metadata byte, and one or more literal data sets corresponding to the sequence of instruction sets and following the sequence of instruction sets in the compressed data fragment; determining a location of the one or more literal data sets in the compressed data fragment using the metadata byte prior to reading the sequence of instruction sets; determining a length of each of the one or more literal data sets based on the sequence of instruction sets; and copying the one or more literal data sets to an in-memory data store, as a batch job for the sequence of instruction sets, based on the location and a corresponding length.