| 摘要 |
A data staging technique when executing batch jobs in a HPC system, combining synchronous and asynchronous data staging processing. In a pre-processor (110a) added to a head node of the HPC system (10), at least one source file for data stage-in, and a target file for data stage-out, both held in permanent storage (30), are identified by analyzing a batch job script (2) input at a user workstation (1). From amounts of data contained in the files, a time required for data stage-in and data stage-out to and from temporary storage (20) are estimated. Data stage-in is set as asynchronous or synchronous based on the estimated time, data stage-out being treated as asynchronous, and each asynchronous data staging processing is further classified as short-term or long-term depending on the estimated time required, each data staging processing being recorded in a data staging management table (44). If a source file is modified, incremental data staging processing is added to the table. With the aid of a data staging list (46) scheduling data staging processing for a plurality of batch jobs, data stage-in of the source file(s) from the permanent storage (30) is performed, monitoring progress of data staging processing in the data staging management table (44), and resources may be allocated for executing the batch job using computing nodes (11) without waiting for data stage-in to complete. The batch job is executed to generate results in the temporary storage (20), and with the aid of a post-processor (110b) at the head node (11), data stage-out is performed to transfer the results to the target file in the permanent storage (30). |
