DATA DE-DUPLICATION SYSTEM

摘要

A data de-duplication system is provided that supports the loading and integration of data from multiple data sources. The data de-duplication system identifies and merges duplicate dimension data records that describe the same entity by creating a single dimension data record that is identified as a single best record ("SBR"). The data de-duplication system further adjusts foreign keys that reference the duplicate dimension data records so that the foreign keys correctly reference the merged dimension data record (i.e., the SBR).