Systems, methods and software for distributed loading of databases

摘要

An information retrieval system comprises a queue for storing load data requests; a set of two or more load servers for retrieving data from a queue and indexing documents for retrieval based on the data retrieved from the queue; and first and second load monitors for monitoring operation of the load servers and providing new data on the queue accessible by the load servers, with the first load monitor configured to provide information regarding its operations to the second load monitor, and the second load monitor configured to respond to status information indicating failure of the first load monitor to take over monitoring of the load servers and providing new data on the queue. Each load server is associated with a service-level-agreement data structure, the data structure including a first field for indicating a minimal priority of document indexing requests that the load server is permitted to accept. The data structure includes a second field for identifying for which databases the load server is permitted to index documents; and the data structure includes a third field indicating a temporal aspect.