Asynchronous processing of messages from multiple search peers

摘要

Systems and methods for asynchronous processing of messages that are received from multiple servers. An example method may comprise: receiving, by a first processing thread, in a non-blocking mode, a plurality of sub-application layer protocol packets from a plurality of servers; processing one or more sub-application layer protocol packets received from a first server of the plurality of servers, to produce a first application layer message; writing the first application layer message to a message queue; processing one or more sub-application layer protocol packets received from a second server of the plurality of servers, to produce a second application layer message; writing the second application layer message to the message queue; and reading, by two or more processing threads of a processing thread pool, two or more application layer messages including the first application layer message and the second application layer message from the message queue, to produce two or more memory data structures based on the read application layer messages.

主权项

1. A method, comprising:
transmitting, by a computer system, a search request to a plurality of search peers of a data aggregation and analysis system; receiving, by a first processing thread of the computer system, in a non-blocking mode, a plurality of sub-application layer protocol packets from the plurality of search peers; parsing one or more sub-application layer protocol packets received from a first search peer of the plurality of search peers, to produce a first application layer message representing a first partial response to the search request; writing the first application layer message to a message queue; parsing one or more sub-application layer protocol packets received from a second search peer of the plurality of search peers, to produce a second application layer message representing a second partial response to the search request; writing the second application layer message to the message queue; and processing, by two or more processing threads of a processing thread pool, two or more application layer messages including the first application layer message and the second application layer message from the message queue, to produce two or more memory data structures representing an aggregated response to the search request.