System and method for shared execution of mixed data flows

摘要

A method, computer program product, and computer system for shared execution of mixed data flows, performed by one or more computing devices, comprises identifying one or more resource sharing opportunities across a plurality of parallel tasks. The plurality of parallel tasks includes zero or more relational operations and at least one non-relational operation. The plurality of parallel tasks relative to the relational operations and the at least one non-relational operation are executed. In response to executing the plurality of parallel tasks, one or more resources of the identified resource sharing opportunities is shared across the relational operations and the at least one non-relational operation.

主权项

1. A computer program product residing on a non-transitory computer readable medium having a plurality of instructions stored thereon which, when executed by a processor, cause the processor to perform operations comprising:
identifying one or more resource sharing opportunities across a plurality of parallel tasks, wherein the plurality of parallel tasks includes one or more relational operations and at least one non-relational operation; executing the plurality of parallel tasks involving the one or more relational operations and the at least one non-relational operation; sharing, in response to executing the plurality of parallel tasks, one or more resources of the identified resource sharing opportunities across tasks involving both the one or more relational operations and at least one non-relational operation, and wherein at least one non-relational operation includes a clustering operation; and designating a task of the plurality of parallel tasks as a primary task, wherein a cluster-id of the primary task is a map output key of a merged task, and wherein in the merged task, cluster-ids of other tasks are part of map output values.