Systems and methods for integrated management of large data sets

摘要

Systems, methods and techniques for managing data sets are described. In particular, systems and methods for modeling, representing, acquiring and disseminating large data sets and data files using metadata are described. The large data sets may include geospatial information. The metadata may be extracted from the data or provided separately. Such metadata may be used to represent the underlying data so as to avoid unnecessary and/or unwanted transfer, exposure or manipulation of the underlying data. The metadata may be represented in a static or dynamic fashion, and may be shared via, for example, social media networks and the like.

主权项

1. A computer-implemented method for disseminating data, comprising:
under the control of one or more computer systems configured with executable instructions, detecting one or more relevant metadata types for a first set of one or more data files of a data provider; extracting metadata of the detected relevant metadata types from the first set, the metadata including at least:
information that identifies the one or more data files of the first set from which the metadata is extracted; andinformation describing one or more relationships between the first set and at least a second set of one or more data files of the data provider, the one or more relationships based on at least one shared characteristic between the first set and the second set; associating the extracted metadata with at least the first set and the data provider; at a time after receiving a browse request from a user to view the first set, determining, based on the browse request and the associated metadata, a subset of the associated metadata that identifies at least the first set, the data provider, and the at least one shared characteristic; responding to the browse request by displaying at least the determined subset of the associated metadata; and at a time after receiving a download request from a user to acquire at least some of the data files of the first set, responding to the request by causing the requested data files of the first set to be transferred from the data provider to the user.