| 摘要 |
Various embodiments of the present invention are directed to the creation of multiple redundant chains of transforms, each on a separate processing thread, for a data flow execution (DFE) of a data transformation pipeline (DTP). For certain of these embodiments, a "distributor" receives a buffer as input and directs that buffer to one of several parallel identical threads to process that buffer. A scheduler would create each of these multiple threads, each thread having an identical (redundant) strings of transforms (chains) downstream from the distributor, and all of which would lead even further downstream to a collector that is responsible for collecting and, if necessary, ordering the buffers processed by the previous redundant chains. In this way, the distributors and collectors provide increased scalability for the pipeline by implicitly partitioning (distributing) individual buffers to one of many threads for at least a part of their execution/processing.
|
