Building a Distributed Dwarf Cube using Mapreduce Technique

摘要

Systems and methods for building a distributed dwarf cube comprising dwarf cuboid using mapreduce technique are disclosed. Data comprising cube values and a cube definition may be received. The cube definition comprises dimensions defined for the cube values. The data received is processed. The data may be transformed to a format. Based upon the format of the data, indexes may be generated. The cube values in one or more dimensions may be sorted based on a cardinality of the cube values. The cube values may be sorted in an order of highest cardinality to lowest cardinality. The cardinality indicates distinctiveness of the cube values in the one or more dimensions. The data may be partitioned into data blocks. A dwarf cuboid may be built for one or more data blocks based upon the order of the cardinality of the cube values.

主权项

1. A method for building a distributed dwarf cube comprising a plurality of dwarf cuboids using a mapreduce technique, the method comprising:
receiving data comprising cube values and a cube definition, wherein the cube definition comprises dimensions defined for the cube values; processing the data:
transforming the data to a format;generating indexes based upon the format of the data;sorting the cube values in one or more dimensions based on a cardinality of the cube values, and wherein the cube values are sorted with in an order of highest cardinality to lowest cardinality, wherein the cardinality indicates distinctiveness of the cube values in the one or more dimensions; partitioning the data into data blocks; and building a dwarf cuboid for one or more data blocks based upon the order of the cardinality of the cube values.