Method and system for dictionary noise removal

摘要

A method and system of removing noise from a dictionary using a weighted graph is presented. The method can include mapping, by a noise reducing agent executing on a processor, a plurality of dictionaries to a plurality of vertices of a graphical representation, wherein the plurality of vertices is connected by weighted edges representing noise. The plurality of dictionaries may further comprise a plurality of entries, wherein each entry further comprises a plurality of tokens. The method can include selecting a subset of the weighted edges, constructing an acyclic graphical representation from the selected subset of weighted edges, and determining an ordering based on the acyclic graphical representation. The selected subset of weighted edges may approximate a solution to the Maximum Acyclic Subgraph problem. The method can include removing noise from the plurality of dictionaries according to the determined ordering.

主权项

1. A method of removing noise from a plurality of dictionaries using a graph having a plurality of vertices and a plurality of edges, the plurality of vertices including a first vertex and a second vertex, the plurality of edges including a first edge connecting the first vertex and the second vertex, the method comprising:
associating dictionaries in the plurality of dictionaries to respective vertices in the plurality of vertices, the associating comprising associating a first dictionary in the plurality of dictionaries to the first vertex and a second dictionary in the plurality of dictionaries to the second vertex, wherein the first dictionary includes a plurality of entries, the plurality of entries including multiple entries each of which comprises multiple words; determining weights for the plurality edges at least in part by determining a first weight for the first edge based on a measure of noise between the first dictionary and the second dictionary; selecting, using the determined weights, a subset of the plurality of edges forming an acyclic graphical representation; determining an ordering based on the acyclic graphical representation; and removing noise from the plurality of dictionaries according to the determined ordering.