HIERARCHICHAL SHARDING OF FLOWS FROM SENSORS TO COLLECTORS

摘要

Systems, methods, and computer-readable media for hierarchichal sharding of flows from sensors to collectors. A first collector can receive a first portion of a network flow from a first capturing agent and determine that a second portion of the network flow was not received from the first capturing agent. The first collector can then send the first portion of the network flow to a second collector. A third collector can receive the second portion of the network flow from a second capturing agent and determine that the third collector did not receive the first portion of the network flow. The third collector can then send the second portion of the network flow to the second collector. The second collector can then aggregate the first portion and second portion of the network flow to yield the entire portion of the network flow.

主权项

1. A method comprising:
receiving, by a first collector device, a first portion of a network flow from a first capturing agent, the first portion of the network flow being captured by the first capturing agent at a first host associated with the first capturing agent; determining, by the first collector device, that a second portion of the network flow was not received, to yield a first determination; based on the first determination, sending, by the first collector device, the first portion of the network flow to a second collector device; receiving, by a third collector device, the second portion of the network flow from a second capturing agent, the second portion of the network flow being captured by the second capturing agent at a second host associated with the second capturing agent; determining, by the third collector device, that the first portion of the network flow was not received, to yield a second determination; based on the second determination, sending, by the second collector device, the second portion of the network flow to the second collector device; determining, by the second collector device, that the first portion of the network flow and the second portion of the network flow are part of a same network flow; and aggregating, by the second collector device, the first portion of the network flow and the second portion of the network flow to yield an aggregated network flow.