Systems and Methods for Memory Optimization Interest-Driven Business Intelligence Systems

摘要

Systems and methods for optimizing memory in an interest-driven business intelligence system in accordance with embodiments of the invention are illustrated. A dictionary for storing values of a dataset may be partitioned in accordance with some embodiments. The partitions of the dictionary may be generated by mapping and reducer processes. The mapping processes receive a value, determine the dimension of the data to which the value belongs, and provides the value to a reducer process that handles the determined dimension. Each reducer process generates partitions of the dictionary for each dimension. The number of values in each partition is determined and compared to a threshold value. Partitions that have a number of values greater than the threshold are stored in a common memory. Partitions smaller than the threshold value can be combined with other partitions such that the cardinality of the combined partition exceeds the threshold value.

主权项

1. An interest-driven business intelligence server system comprising:
at least two processors; and a memory storing an interest-driven business intelligence application wherein the interest-driven business intelligence application directs the at least two processors to:
receive raw data that has a plurality of dimensions;generate at least one partition of a dictionary for each of a set dimensions of the plurality of dimensions of the raw data wherein each at least one partition includes an entry for each of at least a portion of values in one of the plurality dimensions wherein an entry includes a dictionary identifier and a value;determine a number of entries in each of the at least one partitions generated;determine whether the number of entries in each of the at least one partitions is greater than a threshold value;store each of the at least one partitions that have a number of entries greater than the threshold value in a common memory;combine one or more of the at least one partitions having a number of entries less than the threshold value into a partition that includes a number of entries greater than the threshold values; andstore each partitions formed from combining partitions in the common memory.