Elimination of duplicate written records

摘要

A storage device includes first and second buffers. A request to write a new record from a host is received. A hash value (new S) of the new record is calculated. The hash value (new S) of the new record is checked to determine if the hash value exists in a second buffer. If the new S exists in the second buffer, the new record is compared with a record stored in the second buffer corresponding to the new S to check if the new record and the stored record in the second buffer match each other. If the new record and the stored record match each other, a pointer (a record number) is written as write data of the new record to the recording medium. The pointer points to the record already stored in any one of a recording medium and the second buffer.

主权项

1. A deduplication method for eliminating duplication of a record written to a storage device comprising a first buffer, a second buffer, and a recording medium for long-term data storage, the first buffer storing a record number, a hash value, and a write frequency of a record that is recorded for long term storage in the recording medium, the second buffer storing a content of the record in combination with the record number and the hash value, the method comprising the steps of:
receiving a request to write a new record from a host; calculating a hash value (new S) of the new record; checking if the new S exists in the second buffer; if the new S exists in the second buffer such that the new record and a record stored in the second buffer include the same hash value, comparing the new record with the stored record corresponding to the same hash value to determine if the new record and the stored record are the same record; if the new record and the stored record are the same record, writing a pointer (a record number) as write data of the new record to the recording medium, the pointer pointing to the record already stored in one of the recording medium and the second buffer; and if the new record and the stored record include the same hash value and are different records, incrementing a first frequency associated with the new record without incrementing second frequency associated with the stored record.