Systems and methods for facilitating dependency-ordered delivery of data sets to applications within distributed systems

摘要

The disclosed computer-implemented method for facilitating dependency-ordered delivery of data sets to applications within distributed systems may include (1) receiving, at a queue of an application running within a distributed system, a data set from at least one other application running within the distributed system, (2) determining that the data set has a dependency on at least one other data set that has yet to arrive, (3) gating the data set at the queue due at least in part to the dependency, (4) receiving, at the queue, the other data set from the other application, (5) determining that the dependency has been satisfied, and then (6) delivering the data set and the other data set to the application to enable the application to process the data set and the other data set in accordance with the dependency. Various other methods, systems, and computer-readable media are also disclosed.

主权项

1. A method comprising:
receiving, at a queue of an application running on a node within a distributed system, a data set from at least one other application running on another node within the distributed system via an Optimal Flooding Protocol (OFP); obtaining metadata of the data set that is:
described in a domain-specific language; andhoisted outside of the data set; determining, based at least in part on the metadata of the data set, that the data set received from the other application running on the other node has a dependency on at least one other data set that has yet to arrive at the queue of the application, wherein the dependency requires a most up-to-date version of the other data set; gating, due at least in part to the dependency, the data set at the queue of the application running on the node at least until the most up-to-date version of the other data set arrives at the queue of the application running on the node; receiving, at the queue of the application running on the node, the other data set from the other application running on the other node within the distributed system; determining that the dependency has been satisfied based at least in part on receiving the other data set at the queue of the application running on the node; and in response to determining that the dependency has been satisfied, delivering the data set and the other data set to the application running on the node to enable the application to process the data set and the other data set in accordance with the dependency.