SYSTEM AND METHOD OF REDUCING DATA IN A STORAGE SYSTEM

摘要

The system and method of the present disclosure relates to technology for reducing the amount of data stored in a storage system by processing subsets of data stored in data sources using advanced analytics. The process generally includes extracting data from data sources for analysis by ranking the data, marking the data, identifying pattern changes in the data, comparing pattern changes in the data and purging and/or masking the data for storage. The system also includes databases for storing and defining rules, patterns, policies and classification data to be applied to the data from the data sources and analytics to apply the rules, patterns, policies and classification information on the data. As a result, the data stored in the data sources is reduced, and processing efficiency is increased.

主权项

1. A method of reducing data in a storage system, comprising:
accessing the data stored in the storage system by a processor; parsing the data accessed from the storage system into subsets of data using the processor, the parsing comprising categorizing the subsets of data using key identifiers, each of the categorical subsets of data analyzed based on a rule set associated with a respective category for each of the subsets of data; for each of the analyzed subsets of data, using the processor to
detect the subsets of data to be purged based on a threshold condition having been satisfied, and ranking the subsets of data for which the threshold condition has been satisfied, anddetect the subsets of data to be masked based on a policy having been satisfied, and ranking the subsets of data for which the policy has been satisfied; individually marking the subsets of data based on the ranking for purging using the processor when the threshold condition has been satisfied, and individually marking the subsets of data for masking based on the ranking using the processor when the policy has been satisfied; identifying pattern changes using the processor between the subsets of data prior to analysis and the marked subsets of data for purging and between the subsets of data prior to analysis and the marked subsets of data for masking; and processing the subsets of data for permanent change by reducing the amount of data using the processor when pattern changes satisfying a predetermined criteria have been identified, and providing the permanently changed subsets of data with the reduced amount of data to the storage system for storage.