System and method for performing efficient searches and queries in a storage node

摘要

A system and method of providing enhanced data processing and analysis in a storage node, such as a solid state drive (SSD). The SSD includes flash memory and an SSD processing unit capable of executing searches and analysis on the data in the flash memory without returning all of the data to be searched to a host CPU outside of the SSD. Other processing capabilities incorporated into the SSD may include encryption and decryption, compression and decompression, and in-line indexing of data, and data analytics.

主权项

1. A solid state drive comprising:
a solid state drive host interface; a processing unit; and a flash memory; the processing unit being in communication with the flash memory, and comprising: a hardware engine; and a microcontroller; the solid state drive host interface being selected from the group consisting of: PCIe, NVMe, SATA, Fibre Channel, SAS, SCSI over PCIe, and Ethernet, the solid state drive configured to receive a query through the solid state drive host interface, and to return, through the solid state drive host interface, a result of the query, and the hardware engine configured to be controlled by the microcontroller, and to generate a result by applying a processing algorithm to data received from the flash memory, the processing unit configured to execute a search analysis on the processed data, the processed data comprising unstructured data, the search analysis comprising an analysis selected from the group consisting of: comparing the processed data to a specified pattern, wherein the comparing comprises first testing the processed data with a first set of criteria, and second testing the processed data with a second set of criteria in response to the first testing only when the processed data meet the first set of criteria, counting words in the processed data, wherein the counting comprises counting the number of occurrences of a word in the processed data, performing in-line indexing of the processed data, and combinations thereof, wherein the search analysis comprisesthe comparing of the processed data to the specified pattern, wherein the comparing comprises the first testing of the processed data with the first set of criteria, and the second testing of the processed data with the second set of criteria in response to the first testing only when the processed data meet the first set of criteria,
wherein the unstructured data comprises a header and a body, and wherein the first testing comprises first testing the header with the first set of criteria, and the second testing comprises second testing the body with the second set of criteria in response to the first testing only when the header meets the first set of criteria.