Informatica B2B Data Transformation: Basic concepts

Well, I am returning to this topic after a long time. Looking at the stats of this blog, I realized that many people are visiting mainly because of the informatica B2B post here. So, I have decided to continue with it.

In this post, I plan to cover the next level basic components in a Data transformation(DT) project (the file has a format of .cmw). In a bird's eye view, the DT project can comprise of three components: Parser, Mapper, Serializer. 

A Parser is a component which converts raw unstructured (non-relational) data into structured XML data. For ex: It can take an excel file as an input, and put the required excel cell data into the target XML.

A Mapper is a component which helps in mapping the data from one XML structure to another XML structure. This is generally used after the raw unstructured data has been brought into a staging XML structure. Once, the data is in XML, if we need to refine the structure more, we can use a mapper.

A Serializer is a component which prints the XML data into a file of any other format (.txt , don't know about other formats as I have not tried them yet). It does the opposite of a Parser, and can be used anywhere in the project. Sometimes to generate text files to check statuses of few variables or a final text file which contains some information (Ex: Invoice ).

Now, to implement all of the above three components, we need few internal and global components. In any programming language, we use variables, data types, data structures, built in functions to build bigger and customized objects. In the same way, here we make use of "Anchors". As the name suggests, an Anchor is an object which helps keep track of the objects we are dealing with- in this case our input files and XML schema. There are many types of Anchors but the two most basic and most important ones are - Marker and Content. 

A Marker is an anchor which searches for patterns within the input file, and moves the cursor to that position. Say, we are looking for the word "apple" in the input file. Then, if we build a marker which searches for "apple", it will go and find the first "apple" that appears in the file, and place the cursor just after the letter 'e' i.e. after the word apple. This is the default operation of a Marker, but we can customize it to place the cursor in the beginning also. This can be done with the help of Marking property. It can have four values: None (no marking, the cursor remains at the beginning of the file), Begin (begin position), End(end position) and Full(which marks the whole word and places the cursor at the end). 

A Content is an anchor which is used to capture the relevant data between various Marker Anchors. You can imagine the Marker Anchors as borders, and Content Anchors as anything within those borders. The Content Anchor captures the relevant data in the input file and puts it into a target XML structure. This anchor also has the Marking property with the same options as above. This enables you to pick up the same thing again, and put it into different target data elements, as the cursor position remains unchanged. Also, there are Opening and Closing Markers within a content anchor, and they can act as the borders also instead of a separate Marker anchor. But, we need to be careful when we use it, since we should use either the Content's properties or the Marker defined separately. We cannot use the two things together, since the cursor moves with each marker. 

The last thing that I am going to touch base today is the Phase property. It is one of the important concepts in DT. Phase defines the order of execution. There can be three types of phases : Initial, Main, Final. Anchors can have any of these three phases. All the anchors in a particular phase are executed in one go but in the order of their appearance in the code i.e. from top to bottom. As expected, the objects in the initial phase are detected first and then executed in the order of appearance. Similarly,  the main and the final phase are detected next. By default, all Marker Anchors are in initial phase, and all Content Anchors are in main phase. So, the back end engine, first recognizes the Markers and after it has the borders, it searches for the Content in between those borders, only if we want to search for them.

The next post will contain diagrams and will also contain guidelines to start working with Informatica B2B. Happy Learning !