METADATA EXTRACTION, PROCESSING, AND LOADING

摘要

Techniques for data storage are described herein. The techniques may include receiving data 302 having a plurality of the types. Metadata is identified 304 defining the plurality of file types. The techniques include dynamically allocating 306 one or more devices based on the metadata. The techniques include extracting 308 the data at a dynamically allocated device, the extraction based on the metadata, wherein extracting generates secondary metadata. The extracted data is processed 310 at a dynamically allocated device, the processing based on the metadata and secondary metadata. The processed data is loaded 312 from a dynamically allocated device into a data warehouse.

主权项

1. A method comprising:
receiving data having a plurality of file types; identifying metadata defining the plurality of file types; dynamically allocating one or more devices based on the metadata; extracting the data at a dynamically allocated device, the extraction based on the metadata, wherein extracting generates secondary metadata; processing the extracted data at a dynamically allocated device, the processing based on the metadata and secondary metadata; and loading the processed data from a dynamically allocated device into a data warehouse.