FEDERATION OPTIMIZATION USING ORDERED QUEUES

摘要

Methods, systems, and computer program products for optimization of query processing in a data federation system using priority queuing techniques are provided. Priority queuing techniques may include generating a query vector corresponding to a query, comparing the query vector to historical query vectors to determine similarity, determining an expected processing time for the query based on the determined similarity, and inserting the query into a priority ordered queue at a particular position based on the expected processing time.

主权项

1. A federation engine server comprising:
one or more processors; a federated query queue comprising one or more federated queries; a first source query queue corresponding to the first data source, the first source query queue comprising one or more source queries; a data store comprising one or more historical query vectors; a data quality coordinator executable by the one or more processors to:
generate a federated query vector based on a federated query received from a first client of one or more clients;perform a first similarity measure between the federated query vector and at least one of the one or more historical query vectors to determine an estimated processing time of the federated query;set a priority of the federated query based on the estimated processing time of the federated query;based on the priority of the federated query, determine a position of the federated query in the federated query queue relative to at least one of the one or more federated queries;insert the federated query into the federated query queue at the position;generate a plurality of source queries corresponding to the federated query;generate a first source query vector based on a first source query of the plurality of source queries;perform a second similarity measure between the first source query vector and at least one of the one or more historical query vectors to determine an estimated processing time of the first source query;set a priority of the first source query based on the estimated processing time of the first source query;based on the priority of the first source query, determine a position of the first source query in the first source query queue relative to at least one of the one or more source queries;insert the first source query into the first source query queue at the position; andretrieve a data result responsive to the first source query from a first data source in an order that is based upon the position of the first source query in the first source query queue.