SYSTEMS, APPARATUSES, METHODS, AND COMPUTER READABLE MEDIA FOR PROCESSING AND ANALYZING BIG DATA USING COLUMNAR INDEX DATA FORMAT

摘要

Provided are systems, apparatuses, methods and non-transitory computer readable media for efficiently processing and analyzing big data using a columnar index data format. A method of processing big data at a processing system configured as a computer may include generating a dictionary by sorting data based on a column unit of the big data; classifying the sorted data into one or more data blocks for each dictionary based on a data size; generating an index that includes first data values of the respective data blocks in order of the data blocks, for each dictionary; and generating a column ID for each column based on row order of the big data.

主权项

1. A method of processing a big data database at a processing system including at least one computer, the method comprising:
generating, using at least one processor of the at least one computer, at least one dictionary by sorting data of a big data database based on a column unit of the big data database; classifying, using the at least one processor, the sorted data into one or more data blocks for the at least one dictionary based on a desired data size; generating, using the at least one processor, an index that includes first data values of the respective data blocks in an order of the data blocks, for the at least one dictionary; and generating, using the at least one processor, a column ID for each column of the at least one dictionary based on a row order of the big data.