Data Processing Method and System and Client

摘要

A data processing method and system and a client, where a target storage node is determined in a manner of comparing a second vector of received data and first vectors that are corresponding to all storage nodes and prestored on the client that receives the data, and the target storage node no longer needs to be determined in a manner of extracting some fingerprint values as samples from received data and sending the fingerprint values to all storage nodes in a data processing system for query, and waiting for a feedback from the storage nodes.

主权项

1. A data processing method configured to be performed in a data processing system, wherein the data processing system comprises a client, a first storage node and a second storage node, wherein the client is connected to the first storage node and the second storage node, wherein the first storage node is configured to store a first vector representing a feature of data blocks stored in the first storage node, wherein the second storage node is configured to store a second vector that represents a feature of data blocks stored in the second storage node, wherein the client is configured to store the first vector and the second vector, and wherein the data processing method comprises:
receiving, by the client, data; dividing, by the client, the data into multiple data blocks; obtaining, by the client, fingerprint values of the multiple data blocks and a third vector representing a feature of the data received by the client; comparing, by the client, the third vector with the first vector and the second vector to determine the first storage node as a first target storage node; and determining, by the client, non-deduplicate data blocks from the multiple data blocks by comparing the fingerprint values of the multiple data blocks and fingerprint values of the data blocks stored in the first storage node.