SYSTEM AND METHOD FOR DATA CACHING IN PROCESSING NODES OF A MASSIVELY PARALLEL PROCESSING (MPP) DATABASE SYSTEM

摘要

The present technology relates to managing data caching in processing nodes of a massively parallel processing (MPP) database system. A directory is maintained that includes a list and a storage location of the data pages in the MPP database system. Memory usage is monitored in processing nodes by exchanging memory usage information with each other. Each of the processing nodes manages a list and a corresponding amount of available memory in each of the processing nodes based on the memory usage information. Data pages are read from a memory of the processing nodes in response to receiving a request to fetch the data pages, and a remote memory manager is queried for available memory in each of the processing nodes in response to receiving the request. The data pages are distributed to the memory of the processing nodes having sufficient space available for storage during data processing.

主权项

1. A method of managing data caching in processing nodes of a massively parallel processing (MPP) database system, comprising:
maintaining a directory including a list of data pages, the list of data pages stored in one or more data tables, and a storage location of the data pages in the MPP database system; monitoring memory usage in one or more of the processing nodes of the MPP database system by exchanging memory usage information with each of the one or more processing nodes in the MPP database system, each of the one or more processing nodes managing a list of the one or more processing nodes and a corresponding amount of available memory in each of the one or more processing nodes based on the memory usage information; reading data pages from a memory of the one or more processing nodes in response to receiving a request to fetch the data pages; and querying a remote memory manager for available memory in each of the one or more processing nodes in response to receiving a request and distributing the data pages to the memory of one of the one or more processing nodes having sufficient space available for storage during data processing by the one or more processing nodes.