TECHNIQUE FOR REDUCING METADATA STORED IN A MEMORY OF A NODE

摘要

A technique reduces an amount of metadata stored in a memory of a node in a cluster. An extent store layer of a storage input/output (I/O) stack executing on the node stores key-value pairs in a plurality of data structures, e.g., cuckoo hash tables, resident in the memory. The cuckoo hash table embodies metadata that describes an extent and, as such, may be organized to associate a location on disk with a value that identifies the location on disk. The value may be embodied as a locator that includes a reference count used to support deduplication functionality of the extent store layer with respect to the extent. The reference count is divided into two portions: a delta count portion stored in memory for each slot of the hash table and an overflow count portion stored on disk in a header of each extent. One bit of the delta count portion is reserved as an overflow bit that indicates whether the in-memory reference count has overflowed. Another bit of the delta count portion is reserved as a sign bit that indicates whether the value of the remaining delta count portion, which stores the “delta” of the reference count, is positive or negative. Overflow updates to the overflow count portion on disk are postponed until all of the bits of the delta count portion are consumed as negative/positive transitions.

主权项

1. A method comprising:
receiving a first write request having data and receiving a second write request having the data, the write requests processed at a node of a storage system, the node having a processor and a memory, the storage system coupled to a storage device; applying a hash function to the data of the first write request to generate a first hash table index; storing a storage device offset in a slot of a hash table indexed by the hash table index, the hash table stored in the memory; storing the data as an extent on the storage device at the storage device offset, the slot including a first portion of a reference count for the extent; applying the hash function to the data of the second write request to generate a second hash table index identical to the first hash table index; and updating the first portion of the reference count in the slot for the extent, the reference count including a second portion stored on the storage device, wherein the second portion of the reference count has a larger size than the first portion of the reference count.