SCHEDULING AND EXECUTION OF DAG-STRUCTURED COMPUTATION ON RDMA-CONNECTED CLUSTERS

摘要

A server and/or a client stores a metadata hash map that includes one or more entries associated with keys for data records stored in a cache on a server, wherein the data records comprise a directed acyclic graph (DAG), and the directed acyclic graph is comprised of a collection of one or more nodes connected by one or more edges, each of the nodes representing one or more tasks ordered into a sequence, and each of the edges representing one or more constraints on the nodes connected by the edges. Each of the entries stores metadata for a corresponding data record, wherein the metadata comprises a server-side remote pointer that references the corresponding data record stored in the cache. A selected data record is accessed using a provided key by: (1) identifying potentially matching entries in the metadata hash map using the provided key; (2) accessing data records stored in the cache using the server-side remote pointers from the potentially matching entries; and (3) determining whether the accessed data records match the selected data record using the provided key.

主权项

1. An apparatus for accessing data stored in a cache on a server, comprising:
(a) a server or a client storing a metadata hash map that is associated with a cache stored on the server, wherein:
(1) the cache stores one or more data records comprising a directed acyclic graph (DAG), wherein the directed acyclic graph is comprised of a collection of one or more nodes connected by one or more edges, each of the nodes representing one or more tasks ordered into a sequence, and each of the edges representing one or more constraints on the nodes connected by the edges; and(2) the metadata hash map includes one or more entries associated with the data records stored in the cache, each of the entries are associated with a key for a corresponding data record in the cache, each of the entries store metadata for the corresponding data record, and the metadata comprises a server-side remote pointer that references the corresponding data record stored in the cache on the server; (b) the client accessing a selected data record stored in the cache on the server using a provided key by:
(1) identifying one or more potentially matching entries in the metadata hash map using a hash code for the provided key;(2) accessing one or more data records stored in the cache on the server using the server-side remote pointers from the potentially matching entries in the metadata hash map; and(3) determining whether the accessed data records match the selected data record using the provided key; and (c) the client executing the tasks associated with the accessed data record using the constraints associated with the data record.