Loading data with complex relationships

摘要

Data to be loaded into a target system includes entities having corresponding unique primary data value and secondary data values. The secondary data values have associated foreign key relationships, which links the secondary data value to the primary data value of another entity. In loading the data, the entities are split into insertion leaf groups by execution of an iteration process, each iteration involving generation of a respective set of insertion leaf groups containing entities whose secondary data values have no foreign key relationships or only foreign key relationships with the primary data values of entities of the insertion leaf groups generated during any prior iteration. The iteration process is executed until there are no such entities are left. A sequence of generation of the sets of insertion leaf groups is recorded, and the sets of insertion leaf groups are consecutively loaded into the target system according to the sequence.

主权项

1. A computer implemented method for loading data into a target system having a processor, wherein the data comprises entities, each of the entities having a corresponding unique primary data value and one or more corresponding secondary data values, at least a portion of the secondary data values having associated foreign key relationships, each foreign key relationship linking the corresponding secondary data value to the corresponding primary data value of another entity, the method comprising:
splitting the entities into insertion leaf groups by execution of an iteration process by the processor, comprising execution of consecutive iterations, wherein execution of each iteration involves generation of a respective set of the insertion leaf groups, the respective set of the insertion leaf groups containing entities whose secondary data values have either no foreign key relationships or only one or more foreign key relationships with the primary data values of the entities of the insertion leaf groups generated during execution of any prior iteration, wherein the iteration process is executed until there are no entities left whose secondary data values have either no foreign key relationships or only one or more foreign key relationships with the primary data values of the one or more entities of the insertion leaf groups generated during execution of any previous iteration, wherein a sequence of generation of the sets of the insertion leaf groups is recorded; and consecutively loading the sets of the insertion leaf groups of the entities and the foreign key relationships linking their secondary data values with the corresponding primary data values into the target system according to the sequence.