| 摘要 |
A computerized system including a processor and a computer-readable non-transient memory in communication with the processor, the memory storing instructions that when executed manage a novel data structure and related group of algorithms that can be used as a method for representing a set and as a base for very efficient indexing, hash and compression. SHB is an improvement of hierarchical bitmap. An improved database system that can utilize the innovative data structure which includes a raw data stream provided to the system via a data processing module, data blocks, fields indexes tables and a keys table. There is provided an index creating process and a columns creating process, for transforming the data blocks and tables into index blocks and data columns. |
| 主权项 |
1. A computer-implemented method of database management, the method comprising the steps of:
(a) receiving, at a data loader process, one or more rows of values of raw data records; (b) decomposing, at said loader process, each of said one or more rows of values to separate field values, said separation being dictated by a data scheme; (c) assigning a numerical value key id to each said field value of said rows of values; (d) storing in a keys table in a non-transitory computer readable memory one unique instance for each said key id; (e) storing sequentially each said key id in a pre-allocated non-transitory computer readable data memory block, said data memory block including rows divided into data columns according to said data scheme separation, each data column holding a plurality of said key ids of said data column, each row of said rows having a record id indicating a position of said row in said data block within a plurality of data blocks; (f) building field indexes, each said field index related to a data column of said data columns in said data block and including a list of unique instances of each said key id; (g) converting each said field index to a Super Hierarchiel Bitmap (SHB) data structure store in the non-transitory computer readable memory; (h) generating an inverted index block related to said data block including said field indexes and, for each unique instance of each said key id in each said field index, an ordered list of said record ids of said rows in said data block in which said key ids equivalent to said unique instance of said key id are stored; and (i) allocating a new data memory block in said non-transitory computer readable memory when a current record id of a row exceeds a preallocated number of rows in said data memory block. |
