Global commonality and network logging

摘要

Data is divided into blocks, a signature is derived for a block, and the signature is stored in a storage device without retaining the block. The signature may be derived with a hash function. A second signature may be derived for a second block, and compared to the first signature. If there is a match, network data for the second block may be associated to the first signature. If there is not a match, the second signature may be stored, and the second block may be discarded. Policies may be applied, including flagging the data for review, preventing transmission of the data, and storing the data. Network data may be stored with the signatures. Data may be analyzed by dividing it into blocks, deriving a signature for a block, and comparing the signature to stored signatures. If there is a match, network data associated with the matched signature may be retrieved. A plurality of blocks may be compared to the stored signatures to determine degree of commonality.

主权项

1. A method for logging network traffic, the method comprising:
storing a policy specifying triggering a recording of traffic if the traffic is encrypted; receiving a network data stream comprising network packets, the network packets containing packet headers and payloads, at a network monitoring system situated in a data path between a first host and a second host, wherein the network monitoring system is in communication with a non-transitory storage device; extracting, at the network monitoring system, intrinsic data comprising network information from a packet header of a network packet; extracting, at the network monitoring system, extrinsic data from a payload of the network packet; dividing the extrinsic data into a plurality of data blocks; generating a hash signature for individuals of the plurality of data blocks; determining whether a log on the non-transitory storage device contains an identical copy of the hash signature; associating the intrinsic data with the identical copy when the identical copy exists in the log; adding the hash signature to the log and associating the hash signature with the intrinsic data when the identical copy does not exist in the log; determining according to the policy whether the network packet is encrypted or not encrypted; and if the network packet is encrypted, triggering according to the policy a recording of traffic comprising the encrypted network packet, wherein the recorded traffic comprises encrypted content of the traffic, and wherein a decryption key to decrypt the encrypted content is stored in a location apart from the encrypted content and by a third party.