SCALABLE ANALYSIS PLATFORM FOR SEMI-STRUCTURED DATA

摘要

A method of operating a query system includes retrieving objects from a data source. Each of the retrieved objects includes (i) data and (ii) metadata describing the data. The method further includes dynamically creating a cumulative schema. The dynamically creating includes, for each object of the retrieved objects, (i) inferring a schema from the object based on the metadata of the object and inferred data types of elements of the data of the object, (ii) creating a unified schema, and (iii) storing the unified schema as the cumulative schema. The unified schema describes both (a) the object described by the inferred schema and (b) a cumulative set of objects described by the cumulative schema. The method further includes storing the data of each of the retrieved objects in a storage service.