Aggregation framework system architecture and method

摘要

Database systems and methods that implement a data aggregation framework are provided. The framework can be configured to optimize aggregate operations over non-relational distributed databases, including, for example, data access, data retrieval, data writes, indexing, etc. Various embodiments are configured to aggregate multiple operations and/or commands, where the results (e.g., database documents and computations) captured from the distributed database are transformed as they pass through an aggregation operation. The aggregation operation can be defined as a pipeline which enables the results from a first operation to be redirected into the input of a subsequent operation, which output can be redirected into further subsequent operations. Computations may also be executed at each stage of the pipeline, where each result at each stage can be evaluated by the computation to return a result. Execution of the pipeline can be optimized based on data dependencies and re-ordering of the pipeline operations.

主权项

1. A computer implemented method for consistent execution of aggregation expressions, in a non-relational architecture, the method comprising the acts of:
determining, by a computer system, an optimization for execution of an aggregation operation, wherein the aggregation operation includes a plurality of data operations on a non-relational database; modifying, by the computer system, the plurality of data operations to optimize execution; wherein determining the optimization for execution of the aggregation operation includes identifying an aggregation expression to execute as at least part of a data request on the non-relational database; determining, by the computer system, a set of responsive data objects for input into the aggregation expression; flattening, by the computer system, data content stored within each data object based on a specified attribute, wherein the act of flattening includes generating a new data object for each respective data record, stored within an array or as a reference to another data object within a respective data object, matching the specified attribute; executing, by the computer system, the aggregation expression against the flattened data content.