Sparse datatable data structure

摘要

A sparse dataset structure is created by creating column vectors for one or more columns in a dataset that have at least one significant value. Each column vector includes data values for columns of the dataset. Each column vector that is a sparse column vector includes a look-up index array and a value array. Entries in the look-up index array represent columns. The value array includes values for a row in a column. Each entry in the value array points to a row entry in the look-up index array. A side structure includes a row index and a column index. The row index includes a location for an entry for each row where entries point to a location in the column index that identifies a column that has a first significant entry for a row. Alternatively a sparse dataset could be constructed with sparse rows.

主权项

1. In a computing environment a method of creating a representation of a data table having one or more sparse first dimension spaces, the method comprising:
creating a sparse dataset structure by creating a first dimension space vector for each first dimension space in a dataset that has at least one significant value, each first dimension space vector comprising data values for first dimension spaces of the dataset, the sparse data structure comprising one or more sparse first dimension space vectors for sparse first dimension spaces in the dataset, and wherein for each first dimension space vector that is a sparse first dimension space vector, the first dimension space vector includes:
a look-up index array where entries in the look-up index array represent second dimension spaces in the dataset; anda value array where each entry in the value array comprises a value of an entry of a second dimension space in the first dimension space vector, and wherein each entry in the value array corresponds to a second dimension space entry in the look-up index array; creating a side structure, wherein the side structure comprises a second dimension space index and a first dimension space index, wherein the second dimension space index comprises a location for each second dimension space in the dataset where a location can have an entry for a corresponding second dimension space and where an entry for a given second dimension space points to a location in the first dimension space index that has an entry that identifies a first dimension space that has a first significant entry for the given second dimension space, and wherein the first dimension space index comprises an entry for each significant entry in the sparse dataset organized by second dimension spaces.