Decision tree representation for big data

摘要

A method, system, and process for representing a decision tree in a tabular format is discussed. The format may contain all the necessary information to traverse the nodes in parallel on a distributed system while consuming an efficient amount of resources. In some embodiments, the tree may be stored in a relational database as a table.

主权项

1. A method for representing a decision tree in a table, comprising:
receiving a training dataset; building a decision tree from the training dataset, wherein the decision tree comprises a plurality of nodes; storing each node as an individual row in the table on a non-transitory computer readable medium, wherein the row comprises a leftmost child id, a split criterion value (“SCV”), and a path from a root node; distributing the decision tree to multiple nodes in a massive parallel processing (“MPP”) database cluster; receiving a classification dataset to be classified using the decision tree; dividing the classification dataset into a plurality of segments; and distributing the segments to the multiple nodes in the MPP database cluster.