SYSTEM FOR STORAGE, QUERYING, AND ANALYSIS OF TIME SERIES DATA

摘要

A system for storing time series data includes an ingester that prepares metadata indices associated with blocks of incoming time series data and stores the blocks of data in a time series database and the indices in a separate index database. The time series database distributes storage of the data blocks among multiple data nodes. A query layer receives queries and uses the index database to determine which data blocks are needed to process the query, and then requests only those data blocks from the time series database. Processing of the query is performed within the time series database only on those data nodes that contain relevant data, and partial results are passed to an output layer for formation into a final query result.

主权项

1. A system for time series data storage, querying and analysis, comprising:
a data generator that produces time stamped data associated with the behavior of a plurality of assets; an ingester configured to receive the time stamped data from the data generator, the ingester configured to read the received time stamped data and to create a data block and an index associated with the received time stamped data; an index database configured to receive and store the index received from the ingester; a time series database configured to receive the data blocks from the ingester, the time series database storing the data blocks across a plurality of computing devices; a query layer configured to:
receive a query that specifies criteria defining a set of data to be retrieved from the system and an analysis to be performed on that data;request from the index database the indices associated with the data blocks stored in the time series database needed to evaluate the query;prepare a sub-query that will produce appropriate data matching the criteria, the sub-query including the criteria and a logical operation to be performed on data matching the criteria; andsend the sub-query to each of the computing devices that corresponds to the data blocks identified in the request step, above; an evaluator running on each of the plurality of computing devices, the evaluator configured to:
receive the sub-query from the query layer;evaluate the criteria specified in the sub-query with respect to the data blocks stored on the same computing device as the evaluator to select a subset of data; andperform the logical operation specified in the sub-query on the subset of data to produce a sub-result; and an output handler configured to receive the sub-results produced in response to each sub-query and combine them into a query result.