DATA RECORD COMPRESSION WITH PROGRESSIVE AND/OR SELECTIVE DECOMPOSITION

摘要

Disclosed herein are systems and methods for compressing structured or semi-structured data in a horizontal manner achieving compression ratios similar to vertical compression. Collections include structured or semi-structured data include a number of fields and are described using a schema. Fields include information having semantic similarity and are compressed using methods suitable for compressing the type of data. Data of a collection is compressed after fragmentation or may be normalized prior to compression. Data with semantic similarity is compressed using token tables and/or n-gram tables, where higher weighted, consisting of the product of frequency and length, occurring values may be stored in the lower numbered indices of the data table. Records include record descriptor bytes, field descriptor bytes, zero or more array descriptor bytes, zero or more object descriptor bytes, or bytes representing the data associated with the record. Data is indexed or compressed by a suitable module.

主权项

1. A computer-implemented method comprising:
compressing, by a computer associated with a database, one or more data elements using a compression technique, the compression technique compressing the data elements such that each respective data element is individually decompressed when the data element is returned in response to a search query; storing, by the computer, the one or more data elements into a database record comprising one or more data fields, wherein each respective compressed data element is stored into a data field of the record configured to store a type of data element of the respective data element; responsive to the computer receiving a search query requesting a set of one or more data elements stored in one or more records of the database:
querying, by the computer, the database for the set of one or more data elements satisfying the search query; anddecompressing, by the computer, using the compression technique each respective data element in the set of one or more data elements satisfying the search query.