Managing data sets by reasoning over captured metadata

摘要

A data set manager is configured to interact with data processing elements of an information processing system. The data set manager comprises a metadata capture module configured to access or otherwise obtain metadata characterizing data sets associated with the data processing elements, a reasoning module configured to perform one or more reasoning operations on the metadata, and an action recommendation module configured to identify one or more recommended actions for the data processing elements based at least in part on results of the reasoning operations. The metadata characterizes properties of the data sets and relationships among the data sets, and may be defined in accordance with at least one of a specified ontology and a specified class. The data set manager and associated data processing elements may be implemented, by way of example, in cloud infrastructure of a cloud service provider, or on another type of processing platform.

主权项

1. An apparatus comprising:
at least one processing device comprising a memory and a processor coupled to the memory, the at least one processing device implementing a data set manager configured to interact with data processing elements of an information processing system; the data set manager comprising: a metadata capture module configured to obtain metadata characterizing data sets associated with said data processing elements; a reasoning module configured to perform one or more reasoning operations on the metadata; and an action recommendation module configured to identify one or more recommended actions for the data processing elements based at least in part on results of the one or more reasoning operations; wherein the metadata characterizes properties of a given data set and relationships among the given data set and at least one other data set, the properties of the given data set comprising at least one property relating to a history of the data set and at least one property relating to a classification of the data set with respect to a given scheme; wherein the metadata capture module comprises:
a data set discovery module coupled to one or more of the data processing elements and operative to obtain metadata characterizing the data sets therefrom; anda knowledge query and assertion processor operative to process one or more asserted properties or relationships of the given data set in accordance with a model generated by a model manager, the model specifying structural, semantic and format characteristics of the metadata; wherein the model manager is part of a set of common services of the data set manager and is operative:
to add or modify at least one of an ontology and a class utilized by the knowledge query and assertion processor; andto add or modify a policy utilized by a policy engine of the action recommendation module; wherein the knowledge query and assertion processor is operative to process a query or modify knowledge request directed to at least a portion of the metadata; and wherein the action recommendation module comprises a policy engine configured to generate a processing directive to initiate the one or more recommended actions for one or more of the data processing elements based on results of the one or more reasoning operations and at least one policy selected from a policy repository.