Normalization Rule Generation and Implementation Systems and Methods

摘要

Product records having attributes according to various schema are normalized such that the attributes thereof conform to a canonical schema. Attributes for large numbers of product records are normalized according to a map-reduce framework in which mappers are defined but no reducers. Rules are implemented by a rule engine that is instantiated one time per VM of the map-reduce framework. Likewise, each rule may be implemented by only one object instance per rule engine instance. Generation of rules by analysts may be facilitated by defining an attribute hierarchy. A rule defined for a parent attribute may be presented as a default rule for a child attribute and either accepted or revised. Attributes may be clusters and proposed rules generated. Proposed rules may then be accepted or rejected by an analyst.

主权项

1. A method for attribute normalization, the method comprising:
defining a rule engine programmed to create a rule engine instance implementing a single instance of each of a plurality of rule application objects; configuring a cluster of computer systems to implement a map-reduce framework including N mapper instances and zero reducer instances; instantiating a plurality of virtual machines on the cluster of computer systems and instantiating one rule engine instance on each virtual machine of the plurality of virtual machines; receiving a plurality of product records each having a plurality of attributes, the attributes of the product records being represented according to a plurality of different schema; allocating to each rule engine instance an allocation of the plurality of attributes of the plurality of product records to the rule engine instances of the plurality of virtual machines; processing in each rule engine instance the allocation allocated thereto to transform the attributes of the allocation to normalized attributes of a canonical schema; and storing updated records for the plurality of product records including the normalized attributes.