CENTRALIZED PARALLEL BURST ENGINE FOR HIGH PERFORMANCE COMPUTING

摘要

Data storage systems and methods for storing data are described herein. An intermediate storage system is coupled with a High Performance Computing (HPC) system. A plurality of burst engines of the intermediate storage system are directly connected to compute nodes of the HPC. The burst engines are operative to receive file level data from the compute nodes at data transfer rates that at least match to the burst mode output rate of the compute node. The burst engines store the data to high bandwidth storage devices of the intermediate storage system at a data rate that at least matches the burst output rate of the compute node. During a compute phase of the compute node, data is replayed from the intermediate storage devices to a primary storage system at a lower data transfer rate.

主权项

1. A method for transferring data from at least a first compute node of a high performance computing (HPC) system, comprising:
at a compute node of a HPC, capturing file level data output during a burst output; transferring said file level data, via an I/O node of said compute node, at a first data transfer rate to an temporary storage system; receiving said file level data, via and font-end interface of a burst engine of said temporary storage system, at said first data transfer rate; writing said file level data as block level data, via a back-end interface of said burst engine, to a set of storage devices of said temporary storage system, where an aggregate data transfer rate of said set of storage devices equals or exceeds said first data rate.