Storing data across a plurality of storage nodes

摘要

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for storing data on storage nodes. In one aspect, a method includes receiving a file to be stored across a plurality of storage nodes each including a cache. The is stored by storing portions of the file each on a different storage node. A first portion is written to a first storage node's cache until determining that the first storage node's cache is full. A different second storage node is selected in response to determining that the first storage node's cache is full. For each portion of the file, a location of the portion is recorded, the location indicating at least a storage node storing the portion.

主权项

1. A method performed by data processing apparatus, the method comprising:
receiving a file to be stored across a plurality of storage nodes each including a cache; storing the file by storing a plurality of portions of the file each on a different storage node, including writing a first portion to a first storage node's cache; copying the first portion from the first storage node's cache to slower storage of the first storage node, the slower storage of the first storage node being slower than the first storage node's cache; recording, within a log, data specifying the slower storage of the first storage node, the log also specifying a plurality of additional corresponding slower storages of a plurality of storage nodes; determining that the first storage node's cache is full and in response, selecting a different second storage node, wherein selecting the second storage node comprises:
examining the log, andselecting the second storage node as the storage node having a corresponding write time in the log that is the furthest in the past of the write times in the log.