SYSTEMS AND METHODS FOR IN-LINE STREAM PROCESSING OF DISTRIBUTED DATAFLOW BASED COMPUTATIONS

摘要

A data processing system is disclosed that includes machines having an in-line accelerator and a general purpose instruction-based general purpose instruction-based processor. In one example, a machine comprises storage to store data and an Input/output (I/O) processing unit coupled to the storage. The I/O processing unit includes an in-line accelerator that is configured for in-line stream processing of distributed multi stage dataflow based computations. For a first stage of operations, the in-line accelerator is configured to read data from the storage, to perform computations on the data, and to shuffle a result of the computations to generate a first set of shuffled data. The in-line accelerator performs the first stage of operations with buffer less computations.

主权项

1. A machine comprising:
storage to store data; and an Input/output (I/O) processing unit coupled to the storage, the I/O processing unit having an in-line accelerator that is configured for in-line stream processing of distributed multi stage dataflow based computations including for a first stage of operations to read data from the storage and to perform computations on the data with buffer less computations.