System and methods for metadata management in content addressable storage

摘要

Provided is a content addressable storage (CAS) system that allows a user to request, either through an application server or directly to one or more CAS servers, files and content related to a query. In some embodiments, the content can be discovered by searching previously-stored metadata related to each file at the content addressable storage server. The search can also be replicated across multiple content addressable storage servers in order to obtain varied results and redundant results. Duplicate results may be flagged or omitted, and the results are returned to the requester.

主权项

1. A computer-implemented method for managing metadata in a content addressable storage system, the method comprising:
receiving, using one or more computer processors, a file for storage at a first content addressable storage (CAS) server, the file comprising a header and data, and wherein the first CAS server stores data that can be retrieved based on content of the data rather than its storage location or with a hierarchical file system; receiving, using one or more computer processors, the same one or more files for storage at a second CAS server; automatically obtaining, with the one or more computer processors, from the header of the file, metadata associated with the data; storing the metadata in a first metadata storage device, wherein the metadata is stored in association with the data stored in the CAS server; replicating the stored metadata and storing the replicated metadata in a second metadata storage device; receiving, using the one or more computer processors, a query from a requester for content at the CAS server; performing a local search within locally-stored content related to the received query; sending the query to one or more CAS servers; searching beyond a temporary data cache in a local storage device for local content not stored in the CAS server and related to the received query, wherein the local storage device and the CAS server are distinct; sending results of the local search to the requestor; searching the metadata storage device for content related to the received query; and when the metadata associated with the file is indicated by the query: retrieving the file stored in the content addressable storage; and sending the retrieved file to the requester; wherein sending the results of the local search and the retrieved file to the requester further comprises excluding or flagging any duplicate files.