Method and system for extracting shadow entities from emails

摘要

One embodiment provides a system for extracting shadow entities from emails. During operation, the system receives a number of document corpora. The system then calculates word-collocation statistics associated with different n-gram sizes for the document corpora. Next, the system receives an email and identifies shadow entities in the email based on the calculated word-collocation statistics for the document corpora.

主权项

1. A computer-executable method for extracting shadow entities from emails, the method comprising:
loading, by a computing device, a shadow-entity extraction application from storage into memory; executing, by the computing device, the shadow-entity extraction application to perform: receiving a number of document corpora; calculating word-collocation statistics associated with different n-gram sizes for the document corpora; receiving an email; identifying noun phrases in the email; filtering the noun phrases in the email based on the calculated word-collocation statistics of different n-gram sizes for the document corpora, wherein filtering the noun phrases comprises determining a filtering threshold of mutual dependency (MD), point-wise mutual information (PMI), or log-frequency biased mutual dependency (LFMD) value based on an average MD, PMI, or LFMD value of the noun phrases in the email against the MD, PMI, or LFMD value for the document corpora; and identifying shadow entities in the email, based on the calculated word-collocation statistics for the document corpora and the filtered noun phrases in the email.