Performing computations in a distributed infrastructure

摘要

The present invention extends to methods, systems, and computer program products for performing computations in a distributed infrastructure. Embodiments of the invention include a general purpose distributed computation infrastructure that can be used to perform efficient (in-memory), scalable, failure-resilient, atomic, flow-controlled, long-running state-less and state-full distributed computations. Guarantees provided by a distributed computation infrastructure can build upon existent guarantees of an underlying distributed fabric in order to hide the complexities of fault-tolerance, enable large scale highly available processing, allow for efficient resource utilization, and facilitate generic development of stateful and stateless computations. A distributed computation infrastructure can also provide a substrate on which existent distributed computation models can be enhanced to become failure-resilient.

主权项

1. At a fabric based distributed computing architecture including one or more processors and system memory, the distributed computing architecture including a first replica set and at least one second replica set, a method for performing a distributed computation, the method including:
the first replica set performing a first local computation representing a portion of the distributed computation, including:
logically performing a group of operations for the first local computation, including performing at least one output queue related operation;applying physical changes to implement the logical application of each operation in the group of operations, including inserting a message into at least one output queue to physically implement the at least one output queue related operation; andupdating a committed mark within the at least one output queue to indicate that the inserted message is a committed message; transferring the message from the at least one output queue of the first replica set to at least one input queue of the at least one second replica set in accordance with a queue-to-queue protocol that detects the inserted message has been committed; and the second replica set performing a second local computation representing a next portion of the distributed computation, including:
logically performing the group of operations for the second local computation, including performing at least one input queue related operation; andending the second transaction in accordance with one of a transaction commit operation or a transaction abort operation.