DATA QUALITY MONITORING

摘要

A computer implemented method, computer program product and system for data quality monitoring includes measuring a data quality of loaded data relative to a predefined data quality metric. The measuring the data quality includes identifying delta changes in at least one of the loaded data and the data quality rules relative to a previous measurement of the data quality of the loaded data. Logical calculus defined in the data quality rules is applied to the identified delta changes.

主权项

1. A computer-implemented method for data quality monitoring, the method comprising:
measuring, by a computer, a data quality of loaded data relative to a predefined data quality metric, wherein the loaded data is loaded from a data source that includes a data set comprising a plurality of elements, the measuring comprising:
identifying delta changes in the loaded data and data quality rules relative to a previous measurement of the data quality of the loaded data;applying a logical calculus defined in the data quality rules to the identified delta changes;decomposing a selected quality rule into single terms and projecting the terms onto a tree structure with each node of the tree structure corresponding to one individual term of the selected quality rule; andapplying each individual data column to the tree structure resulting in an intermediary column cache for each node containing an intermediate result of a selected operation.