SYSTEM AND METHOD FOR DATA TRANSFORMATION APPLICATIONS

摘要

A system and method for developing dataflow applications where a dataflow transformation is encapsulated into a dataflow graph expressed as a reusable map component using a host development language such as Java, and the dataflow graphs are linked together via ports. A map component may employ dynamic logic where internal sub-graphs or links are added or properties changed based on information received from the compiler, such as port types, design properties, or static data transformation. The dataflow application may be executed in a parallel processing environment by recognizing the linked data processes within the map components and assigning threads to the linked data processes. The execution environment monitors threads, detects and corrects deadlocks, logs and controls program exceptions and processes data input and output ports of the map components in parallel to take advantage of data partitioning schemes. Port implementation supports multi-state null value tokens to more accurately report exceptions.