Systems and methods for multicore processing of data with in-sequence delivery

摘要

Methods and systems are described to allow for the parallel processing of packets and other subsets of data that are to be delivered in order after the completion of the parallel processing. The methods and systems may process packets and subsets of data that may vary in size by orders of magnitude. The packets may be transmitted and/or received over data transmission networks that may be orders of magnitude faster than the processing speeds of the parallel processors. Entire packets or subsets of data may be allocated to individual processing units without segmenting the packets between the processing units. A count value may be inserted as metadata to received packets in order to indicate a relative order of arrival. The metadata may be utilized by a multiplexor at the output of the parallel processing units in order to maintain in-sequence delivery of the processed packets.

主权项

1. A data processing appliance (DPA) comprising:
an input configured to receive a plurality packets; a demultiplexor configured to append metadata to the plurality of packets, wherein the metadata indicates an order in which the plurality of packets were received; a plurality of processing cores, each of the plurality of processing cores configured to process a portion of the plurality of packets independent of the other processing cores; and a multiplexor configured to:
multiplex the plurality of packets after the plurality of packets are output from the plurality of processing cores based on the appended metadata, andprior to sending a given packet to an output, ensure the output of the given packet is delayed by at least a predetermined amount of time relative to an arrival time of the given packet using metadata corresponding to the given packet, wherein the metadata corresponding to the given packet comprises a timestamp value, and the predetermined amount of time exceeds an expected maximum processing delay for a maximum packet size given the processing task implemented by the plurality of processing cores.