| 摘要 |
Some embodiments of the invention employ algorithms enabling the calculation of one or more statistical moments in a single pass of a dataset. For example, some embodiments may apply algorithms for calculating statistical moments to a dataset using a map-reduce framework, whereby an input dataset is partitioned into multiple shards, a separate map process is used to apply an algorithm enabling calculation of one or more statistical moments in a single scan to each shard, and one or more reduce processes consolidate the results generated by the map processes to calculate the one or more statistical moments across the entire dataset. In other embodiments of the invention, a map-reduce framework may be employed to apply algorithms enabling calculation of a covariance between data elements expressed in a dataset, instead of or in addition to one or more statistical moments.
|
