User defined data partitioning (UDP)—grouping of data based on computation model

摘要

Methods, systems, and computer program products are provided for generating application-aware data partitioning to support parallel computing. A label for a user defined data partitioning (UDP) key is generated by a labeling process to configure data partitions of original data. The UDP is labeled by the labeling process to include at least one key property excluded from the original data. The data partitions are evenly distributed to co-locate and balance the data partitions and corresponding computations performed by computational servers. A data record of the data partitions is retrieved by performing an all-node parallel search of the computational servers using the UDP key.

主权项

1. A computer system for data partitioning, the computer system comprising:
a memory; and a computer processor to:
generate a user defined data partitioning key to configure data partitions of original data, the user defined data partitioning key generated based upon a computational model applied to the original data, the user defined data partitioning key to include at least one key property of the computational model, the at least one key property is excluded from the original data, and the user defined data partitioning key is generated or learnt from the original data based on an application; andallocate the data partitions to co-locate the data partitions and corresponding processing of computations associated with the computational model.