A METHOD AND SYSTEM FOR INTEGRATING DATA INTO A DATABASE

摘要

A method and system for integrating data into a database (6) comprises storing data from a plurality of data sources (S1,Si). The system comprises a rule learning module (1 ) and a duplicate elimination module (2). The rule learning module (1 ) operates in an initial rule learning stage. The duplicate elimination module (2) then operates in a de-duplication stage using the learnt rules. The de-duplication rules use conditional probability to determine the probability of records in the data sources (S1,Si) being duplicates of one another. Duplicate records are integrated and stored in the integrated database (6).