A METHOD AND PROCESS FOR ENABLING DISTRIBUTING CACHE DATA SOURCES FOR QUERY PROCESSING AND DISTRIBUTED DISK CACHING OF LARGE DATA AND ANALYSIS REQUESTS

摘要

A method and system for large data and distributed disk cache processing in a Pneuron platform (100). The system and method include three specific interoperable but distributed functions: the adapter/cache Pneuron (14) and distributed disk files (34), a dynamic memory mapping tree (50), and distributed disk file cleanup (28). The system allows for large data processing considerations and the ability to access and acquire information from large data files (102) and rapidly distribute and provide the information to subsequent Pneurons (104) for processing. The system also provides the ability to store large result sets, the ability to deal with sequential as well as asynchronous parallel processing, the ability to address large unstructured data; web logs, email, web pages, etc., as well as the ability to handle failures to large block processing.