Method and apparatus for accelerated format translation of data in a delimited data format

摘要

Various methods and apparatuses are described for performing high speed format translations of incoming data, where the incoming data is arranged in a delimited data format. As an example, the data in the delimited data format can be translated to a fixed field format using pipelined operations. A reconfigurable logic device can be used in exemplary embodiments as a platform for the format translation.

主权项

1. A method comprising:
receiving data in a delimited data format, wherein the received data in the delimited data format comprises (1) a plurality of data characters arranged in a plurality of fields, (2) a plurality of shield characters, and (3) a plurality of field delimiter characters, the field delimiter characters defining a plurality of boundaries between the fields; converting the received data to a fixed field format, wherein the converting step comprises the reconfigurable logic device (1) distinguishing between field delimiter characters and data characters in the received data based on the shield characters, (2) identifying the fields in the received data based on the field delimiter characters, and (3) arranging the data characters sharing the same identified fields into fixed-size fields such that the converted data comprises the data characters in the fixed-size fields stripped of the field delimiter characters and the shield characters; performing a plurality of processing operations on the converted data to generate processed data in the fixed field format; loading the processed data into a database; and wherein the converting step is performed by a reconfigurable logic device.