Parallel filing system using user level data storage

摘要

A storage system is used with a parallel processing system comprising a plurality of nodes 1. An application 3 is split into individual applications and distributed across the nodes. The storage system splits files into a plurality of fragments. The pieces of the files are stored on a plurality of storage devices 5 as files in the native filing system of the nodes. The application is provided with the information 6 about the distribution of the file fragments. I/O processes 4 are launched by the application on the nodes which access the storage devices. Message Passing Interface (MPI) messages are used to communicate with the I/O processes. Data in the files may be transformed to and from a format native to the machine when the file is accessed. The format may be the endianness of the data.