LEVERAGING HARDWARE ACCELERATORS FOR SCALABLE DISTRIBUTED STREAM PROCESSING IN A NETWORK ENVIRONMENT

摘要

An example method for leveraging hardware accelerators for scalable distributed stream processing in a network environment is provided and includes allocating a plurality of hardware accelerators to a corresponding plurality of bolts of a distributed stream in a network, facilitating a handshake between the hardware accelerators and the corresponding bolts to allow the hardware accelerators to execute respective processing logic according to the corresponding bolts, and performing elastic allocation of hardware accelerators and load balancing of stream processing in the network. The distributed stream comprises a topology of at least one spout and the plurality of bolts. In specific embodiments, the allocating includes receiving capability information from the bolts and the hardware accelerators, and mapping the hardware accelerators to the bolts based on the capability information. In some embodiments, facilitating the handshake includes executing a shadow process to interface between the hardware accelerator and the distributed stream.

主权项

1. A method, comprising:
allocating a plurality of hardware accelerators to a corresponding plurality of bolts of a distributed stream in a network, wherein the distributed stream comprises a topology of at least one spout and the plurality of bolts; facilitating a handshake between the hardware accelerators and the corresponding bolts to allow the hardware accelerators to execute respective processing logic on data elements of the distributed stream according to the corresponding bolts; and performing elastic allocation of the hardware accelerators and load balancing of stream processing in the network.