Metadata optimization for network replication using representative of metadata batch

摘要

In one embodiment, in response to a list of a plurality of fingerprints representing data chunks to be replicated from a source storage system to a target storage system over a network, one of the fingerprints is selected as a representative fingerprint that matches selection criteria. The source storage system generates a fingerprint representation for each of the fingerprints that is smaller than the corresponding fingerprint. The representative fingerprint and the fingerprint representations are transmitted to the target storage system over the network, without transmitting all full fingerprints of the data chunks, to allow the target storage system identifying which of the data chunks are missing. In response to information received from the target storage system indicating one or more data chunks that are missing at the target storage system, the missing data chunks are then transmitted to the target storage system.

主权项

1. A computer-implemented method for replicating data over a network, the method comprising:
in response to a list of a plurality of fingerprints representing a plurality of data chunks to be replicated from a source storage system to a target storage system over a network, selecting one of the fingerprints as a representative fingerprint that matches a selection criteria; generating, at the source storage system, a fingerprint representation for each of the plurality of fingerprints, wherein a fingerprint representation contains only a portion of the corresponding fingerprint, and wherein the representative fingerprint is a full fingerprint; transmitting the representative fingerprint and the fingerprint representations to the target storage system over the network, without transmitting all full fingerprints of the data chunks, to allow the target storage system identifying which of the data chunks are missing; receiving a response from the target storage system in response to the representative fingerprint and the fingerprint representations, the response including a bitmask having a plurality of bits each indicating whether one of the fingerprints is missing at the target storage system; in response to the response, transmitting one or more fingerprints that are missing to the target storage system based on the bitmask, such that a network bandwidth required to transmit the fingerprints of the data chunks from the source storage system to the target storage system over the network is reduced; and in response to information received from the target storage system indicating one or more data chunks that are missing at the target storage system, transmitting the missing data chunks to the target storage system.