COLUMN BASED DATA TRANSFER IN EXTRACT TRANSFORM AND LOAD (ETL) SYSTEMS

摘要

Executing a plurality of transform stages in an extract, transform and load (ETL) job including, for each of the transform stages, receiving a plurality of input row identifiers (RIDs) corresponding to a first plurality of source database table rows in a source database table. Data is retrieved directly from a subset of the source database table columns in the first plurality of source database table rows based on the input RIDs and transform logic. Partial row data including data from the subset of the source database table columns is generated for each of the first plurality of source database table rows. Transformed data is generated based on the partial row data and to the transform logic. Output RIDs corresponding to a second plurality of rows in the source database table that include a least a subset of the transformed data are output to a downstream stage.