ANALYZING MULTIPLE DATA STREAMS AS A SINGLE DATA OBJECT

摘要

Embodiments of the present invention allow multiple data streams to be analyzed as a single data set. The single data set may be described as a stream set herein. The multiple streams that are included in the stream set may be specified through a user script or query. For example, a query may be used to gather all streams created within a date range. The query could include one or more filters to gather certain information from the data streams or to exclude certain data streams that otherwise are in the query's range. A stream may be an unstructured byte stream of data. The stream may be created by append-only writing to the end of the stream. The stream could also be a structured stream that includes metadata that defines column structure and affinity/clustering information.

主权项

1. One or more computer-storage media having computer-executable instructions embodied thereon that when executed by a computing device perform a method of generating a stream set that includes data from multiple data stream files, the method comprising:
receiving a query specifying a virtual column and a range of virtual column values, the range bound by a high value and a low value; generating a plurality of virtual column values within the range; generating a plurality of potential stream names by substituting individual values from the plurality of virtual column values into a stream name template; generating a plurality of available data streams by determining whether names in the plurality of potential stream names correspond to names of actual data streams within a computer-storage location; and generating a stream set by reading data from the plurality of available data streams.