PARALLEL LOG STRUCTURED FILE SYSTEM COLLECTIVE BUFFERING TO ACHIEVE A COMPACT REPRESENTATION OF SCIENTIFIC AND/OR DIMENSIONAL DATA

摘要

Collective buffering and data pattern solutions are provided for storage, retrieval, and/or analysis of data in a collective parallel processing environment. For example, a method can be provided for data storage in a collective parallel processing environment. The method comprises receiving data to be written for a plurality of collective processes within a collective parallel processing environment, extracting a data pattern for the data to be written for the plurality of collective processes, generating a representation describing the data pattern, and saving the data and the representation.