Scrubbe to remove personally identifiable information

摘要

A personally identifiable information (PII) scrubbing system. The PII scrubbing system surgically scrubs PII form a log based on a scrubber configuration corresponding to the log. The scrubber configuration includes context information about locations and types of PII in the log and rules specifying how to locate and protect the PII. Scrubber configurations are quickly and easily created or modified as scrubbing requirements change or new scenarios are encountered. The flexibility provided by the scrubber configurations allows only the PII to be scrubbed, even from unstructured data, without having to include surrounding data. Many consumers can use the scrubbed data without needed to expose the PII because less non-personal data is obscured. Surgical scrubbing also retains the usefulness of the underlying PII even while protecting the PII. Consumers can correlate the protected PII to locate specific information without having to expose additional PII.

主权项

1. A method of scrubbing a data set having messages containing both non-personal data and personally identifiable information, the method comprising:
loading a message containing both non-personal data and personally identifiable information; loading a scrubber configuration containing a rule set for scrubbing the data set; parsing the message into fields based on the rule set, wherein unstructured data fields are formatted and delimiters are added to the unstructured data field such that personally identifiable information is identifiable from unlabeled data; scrubbing only the personally identifiable information in the message based on the rule set to produce a scrubbed message, the personally identifiable information being associated with metadata that identifies a type of personally identifiable information, and applying a corresponding scrubbing rule to the type of personally identifiable information, the corresponding scrubbing rule including:
generating replacement values for the personally identifiable information in the message based on the rule set, including generating a replacement value for a first instance of specific personally identifiable information in the message based on the corresponding scrubbing rule and storing a reference to the replacement value associated with the specific personally identifiable information; andsubstituting replacement values for the personally identifiable information in the message to create the scrubbed message, including retrieving the replacement value associated with the specific personally identifiable information using the reference when additional instances of the specific personally identifiable information are encountered, and using the retrieved replacement value for the additional instances of the specific personally identifiable information; and saving the scrubbed message.