Introduction
Legacy read connectors often have a two-stage process, which includes a Reading stage and a Parsing stage. These stages help to read byte streams from the source and convert them into records or data that can be used in downstream applications. In this article, we will discuss the differences between the Reading stage and the Parsing stage in a legacy read connector.
The Reading Stage
The Reading stage is the initial step in a legacy read connector. Its primary function is to read byte streams from the source and write them out to a Byte Container. During this stage, the connector reads the data from the source and saves it to a temporary location. The Reading stage continues until all the data has been read from the source.
The Parsing Stage
The Parsing stage is the second stage in a legacy read connector. Its primary function is to convert the byte stream into records or data that can be used in downstream applications. There are two options available for the Parsing stage depending on the Reading stage:
- If the Reading stage has already completed, the Parsing stage reads bytes from the Byte container, parses them and writes them to the Records container.
- If the Reading stage has not completed, the Parsing stage reads bytes from the source, parses them, and writes them to the Records container.
Connection-Based Connectors
Unlike legacy read connectors, connection-based connectors do not separate the Reading and Parsing stages. The reading and parsing processes are combined into a single stage, which reads data from the source and immediately parses it into records or data that can be used in downstream applications.
Conclusion
The Reading stage and Parsing stage are two crucial components of a legacy read connector. The Reading stage reads data from the source and saves it to a temporary location, while the Parsing stage converts the byte stream into records or data that can be used in downstream applications. In connection-based connectors, these stages are combined into a single stage. Understanding the differences between the Reading and Parsing stages can help you build more efficient and effective data connectors.
Comments
0 comments
Please sign in to leave a comment.