Optimizing data processing using dynamic schemas

摘要

A computer system accesses rows of feed data and converts the received feed data into portions of binary blob data. The computer system also sends the binary blob data to a database server which is configured to access metadata associated with a feed including a dynamic server statement to determine how to convert the binary blob data to a server table with a blob column configured to store the rows of feed data. The database server accesses feed data belonging to a particular feed and executes a dynamic server statement to create a relational dataset in an in-memory table of the server. A second dynamic statement applies data processing conditions indicated in the metadata. When feed data rows match conditions, the computer system places feed data row information into an alert table that includes references to the blob table with blob data, thereby triggering an alert.

主权项

1. A method, implemented at a computer system that includes one or more processors, for efficiently uploading data to a database using dynamic schemas, the method comprising:
an act of receiving from an application server one or more portions of binary blob data at a database server; an act of executing a dynamic server statement to store the binary blob data into a database table, wherein the dynamic server statement is stored within metadata associated with an original feed; an act of restoring one or more feed data rows from the binary blob data by substituting varying size references with the actual feed data, while keeping the feed data in a binary format; and an act of storing the restored feed data in the database table in a blob data column, wherein each feed data row results in one binary blob value in one corresponding row in the database table.