Implementation of clustered in-memory database

摘要

An in-memory database system and method for administrating a distributed in-memory database, comprising one or more nodes having modules configured to store and distribute database partitions of collections partitioned by a partitioner associated with a search conductor. Database collections are partitioned according to a schema. Partitions, collections, and records, are updated and removed when requested by a system interface, according to the schema. Supervisors determine a node status based on a heartbeat signal received from each node. Users can send queries through a system interface to search managers. Search managers apply a field processing technique, forward the search query to search conductors, and return a set of result records to the analytics agents. Analytics agents perform analytics processing on a candidate results records from a search manager. The search conductors comprising partitioners associated with a collection, search and score the records in a partition, then return a set of candidate result records after receiving a search query from a search manager.

主权项

1. A computing system hosting an in-memory database, the system comprising:
a partitioner node comprising a processor configured to, in response to receiving a collection of one or more records of the in-memory database, determine whether to compress the collection based on a machine-readable schema file associated with the collection, logically partition the collection into one or more partitions according to the schema file, and distribute the one or more partitions to one or more storage nodes according to the schema file; a storage node comprising a non-transitory machine-readable main memory storing a partition of the in-memory database received from one or more partitioner nodes associated with the storage node according to the schema file; a search manager node comprising a processor configured to receive a search query from a client device of the system, and transmit the search queries as search conductor queries to one or more search conductors upon receiving the search query from the client device, wherein the search query is a machine-readable computer file containing parameters associated with one or more records satisfying the search query, and wherein the search manager node transmits the search queries to the one or more search conductors in accordance with the schema file; a search conductor node associated with one or more partitioners according to the schema file and comprising a processor configured to, in response to receiving a search conductor query from the search manager node: query a set of one or more partitions of the in-memory database as indicated by the search conductor query, identify one or more candidate records of the in-memory database stored in the set of partitions queried by the search conductor, calculate a first score for each respective candidate record using a scoring algorithm, and transmit to the search manager node a set of one or more query results containing one or more candidate records satisfying a threshold value; and an analytics agent node comprising a processor configured to automatically generate a machine-readable computer file containing a set of one or more results derived from the set of query results, responsive to identifying in the set of query results received from the search manager node, wherein each of the storage node, the search conductor node, and the analytics agent node is a distinct node.