Cache population optimization for storage gateways

摘要

Methods and apparatus for WAN-optimized cache population at a storage gateway appliance are disclosed. A read request is received at the appliance from a client. The request is directed to a particular data block of a data chunk of a storage object of a remote storage service. A method comprises, in response to the read request, initiating a first data transfer of the data chunk from the remote storage service to an intermediate device. The method includes determining, based on one or more criteria, contents of one or more other data transfers including a second data transfer from the intermediate device to the appliance, initiating the second data transfer to transmit the requested data block to the appliance, and providing the particular data block to the client from the appliance.

主权项

1. A method, comprising:
in response to receiving a read request at an intermediate device from a storage appliance that uploads data blocks to a remote storage service from a client network comprising the storage appliance and a client, wherein the read request from the storage appliance was sent in response to a read cache miss resulting from a read request received at the storage appliance from the client, wherein the read request was directed to at least a particular data block of a particular data chunk of a storage object of the remote storage service, wherein the storage object comprises a plurality of data chunks, and wherein each data chunk comprises a plurality of data blocks: initiating a first data transfer of the particular data chunk from the remote storage service to the intermediate device, wherein the particular data chunk is transferred over a first network path within a provider network comprising the remote storage service and the intermediate device, and wherein the first network path has a first bandwidth capacity; subsequent to said initiating, determining, at the intermediate device and based on one or more criteria, contents of one or more other data transfers, including a second data transfer, from the intermediate device to the storage appliance over a second network path having a second bandwidth capacity, wherein the contents of the one or more other data transfers include data distinct from the particular data block that was requested, and wherein a criterion of the one or more criteria used to determine the contents is based at least in part on one or more of: a characteristic of input/output requests received at the storage appliance, network congestion, or one or more metrics of resource usage at the intermediate device or storage appliance; and sending, by the intermediate device to the storage appliance, at least the particular data block and the distinct data as part of the second data transfer, wherein at least the particular data block is sent to the storage appliance for transfer within the client network comprising the storage appliance and the client.