Informatica B2B Data Transformation: Basic concepts (working with Parser)

Below, is a diagram which shows how a basic Parser looks like, and also the Marker and Content Anchors. It also shows what are the various ways of looking in a file and searching for a pattern in the file.

In order to open a new blank project in DT, go to File -> New -> Project -> Data Transformation Project -> Blank project. Give the project a name, and then you are ready to go. Initially you will have nothing but a blank code file (.tgp file) under the Scripts section on the left hand side (Data transformation Explorer). If you are not able to see the Data transformation Explorer, go to Window -> Show View -> Data Transformation Explorer.

Before you begin anything, just prepare a basic XSD file (preferably using XML Spy) which contains a Root element and a Data element as the child element of that Root.

Initial Stage (Only a blank code file)

Open the image files in a new tab to view it in a better way. 

A simple XSD file

It is better to use a software such as XML Spy to make a Schema definition file. But, if you are well versed with XML and XSD, you can directly code it here.

The basic components of a Parser

It is an empty Parser. If we look at the image, we will see the various properties of a Parser. We will go deep into each one of them, in the coming posts. Example source is a required element. It is the input to the parser.

I used a simple Text as an input

Other input options are also available, but for simplicity I used a Text input. After doing this Press Ctrl-F10, and you will see the input on the right hand side  as shown in the next image.

Marker in Action

You can see that once I introduce a Marker, which searches for "apple" in the body of the code, the "apple" word in the input file gets marked. You can use the property expander/minimizer to view the other properties of a marker (Phase, Marking etc).

Note: You can enter a Marker by either selecting the word from the right hand side and selecting "insert Marker" from the right click, or you can just start typing in Marker, and press Enter in the code area.

Content Anchor

Next, if we add a content anchor, by just typing in Content in the code area below the marker, we will see that the text "very much" gets selected in a different color. This denotes the part to be captured and put into the XSD element. For now, the data holder of the Content is blank, but we will add it eventually, otherwise the project will be invalid. Also, note the disable and optional features, which helps you to discard parts of code while executing the code.

Marker Content Maker

In this, we can see how the Markers act as a border for the content. We define one more Marker, which searches for "much", just below the Content anchor and hence we see that the scope of the Content anchor is then limited to the word "very" (along with spaces). Next, we will place this word into a data element in the target XSD.

XSD element in data holder

When we click on the data holder of the Content Anchor, we will find that a new pop up box opens up. This shows the variables schema that are available in the project. It also shows the local variables which contains the system variables as well any custom variables that you might have made. For now, my XSD is placed in no target namespace. So I selected the Data element from there.


Once you done with the above steps, save the code and press Ctrl-F10 to ensure that the markings on the input file are right. Also, if you want to experiment with properties of Marking etc, you can do that. You will understand the concept better by experimenting with the options available. Anyways, now to run the project, you can go to Run in the menu bar or press F5 or Ctrl-F5. Once you do that, you will see the Events tab filled with the execution steps below the code window, and also an output file under the Results section on the left hand side. Open the output file to view the desired XML file.

The final output file

You can browse through the execution in the events tab, and understand how the back end engine works. You will also observe that the Markers are executed first as they are in the initial phase followed by the Content which is in the main phase. Also, you will see that the Data element contains the word "very", hence verifying that the code works fine.


This post showed the working of a Parser. The example here is very basic, but is the foundation of more complex things that can be handled in a parser in DT. Please note the important concepts in here: Marking property, Phase property, Data holder, Opening and Closing Marker.
If you happen to pass by, and have any doubts, then please drop a comment to this post, and I will get back to you.

Informatica B2B Data Transformation: Basic concepts

Well, I am returning to this topic after a long time. Looking at the stats of this blog, I realized that many people are visiting mainly because of the informatica B2B post here. So, I have decided to continue with it.

In this post, I plan to cover the next level basic components in a Data transformation(DT) project (the file has a format of .cmw). In a bird's eye view, the DT project can comprise of three components: Parser, Mapper, Serializer. 

A Parser is a component which converts raw unstructured (non-relational) data into structured XML data. For ex: It can take an excel file as an input, and put the required excel cell data into the target XML.

A Mapper is a component which helps in mapping the data from one XML structure to another XML structure. This is generally used after the raw unstructured data has been brought into a staging XML structure. Once, the data is in XML, if we need to refine the structure more, we can use a mapper.

A Serializer is a component which prints the XML data into a file of any other format (.txt , don't know about other formats as I have not tried them yet). It does the opposite of a Parser, and can be used anywhere in the project. Sometimes to generate text files to check statuses of few variables or a final text file which contains some information (Ex: Invoice ).

Now, to implement all of the above three components, we need few internal and global components. In any programming language, we use variables, data types, data structures, built in functions to build bigger and customized objects. In the same way, here we make use of "Anchors". As the name suggests, an Anchor is an object which helps keep track of the objects we are dealing with- in this case our input files and XML schema. There are many types of Anchors but the two most basic and most important ones are - Marker and Content. 

A Marker is an anchor which searches for patterns within the input file, and moves the cursor to that position. Say, we are looking for the word "apple" in the input file. Then, if we build a marker which searches for "apple", it will go and find the first "apple" that appears in the file, and place the cursor just after the letter 'e' i.e. after the word apple. This is the default operation of a Marker, but we can customize it to place the cursor in the beginning also. This can be done with the help of Marking property. It can have four values: None (no marking, the cursor remains at the beginning of the file), Begin (begin position), End(end position) and Full(which marks the whole word and places the cursor at the end). 

A Content is an anchor which is used to capture the relevant data between various Marker Anchors. You can imagine the Marker Anchors as borders, and Content Anchors as anything within those borders. The Content Anchor captures the relevant data in the input file and puts it into a target XML structure. This anchor also has the Marking property with the same options as above. This enables you to pick up the same thing again, and put it into different target data elements, as the cursor position remains unchanged. Also, there are Opening and Closing Markers within a content anchor, and they can act as the borders also instead of a separate Marker anchor. But, we need to be careful when we use it, since we should use either the Content's properties or the Marker defined separately. We cannot use the two things together, since the cursor moves with each marker. 

The last thing that I am going to touch base today is the Phase property. It is one of the important concepts in DT. Phase defines the order of execution. There can be three types of phases : Initial, Main, Final. Anchors can have any of these three phases. All the anchors in a particular phase are executed in one go but in the order of their appearance in the code i.e. from top to bottom. As expected, the objects in the initial phase are detected first and then executed in the order of appearance. Similarly,  the main and the final phase are detected next. By default, all Marker Anchors are in initial phase, and all Content Anchors are in main phase. So, the back end engine, first recognizes the Markers and after it has the borders, it searches for the Content in between those borders, only if we want to search for them.

The next post will contain diagrams and will also contain guidelines to start working with Informatica B2B. Happy Learning !

Informatica B2B Data Transformation: First Look

Although I am pretty much new to this technology, but still I would like to share what I know about it, mainly because there is very less help material available on the net. I recently switched jobs, and my new job is in a start up with offices at London, New York and Mumbai. The first thing that I got trained in, and part of which is going on is on the Informatica B2B technology (B2B Data transformation and B2B Data Exchange). As per my knowledge, Informatica bought a company/product named Complex Data Exchange (CDE) seven or eight years ago and then integrated it to the main tool as B2B technology. This technology is very new in the market, I have hardly come across any developers and users yet, but it is on the rise, and slowly many businesses are starting to understand the importance of it. 

Informatica Power Centre, as we know, is useful when the data is in the form of tables, or more generally, structured data. Informatica B2B Data transformation is a tool which deals with unstructured data. When I say unstructured, it doesn't necessary mean that there is no logic at all in the source file. The thing is, the structure and logic present in the source file cannot be handled by Data management tools such as Informatica or DataStage (not sure about DataStage though). This feature helps the business for better data integration across departments and regions. The formats which are generally handled by B2B are xml, doc, xls, HL7, HIPAA, SWIFT, etc. The back-end of this tool is coded in JAVA, and thus can be altered with add on libraries (thought till now I have not felt the need of it).

This is how a normal working screen looks like. On the right side, we can see the source file on which the operations take place (format here is xml). On the left, we have the Data transformation Explorer, which gives us the information about the projects and its components (source file, script (actual code), XSD (xml schema definition), Results (output) and also additional files if required). Below, we see the various options such as Help View, Events view, Schema view etc. We will go in deeper on each of the features later, but the point to be noted here is the Events view tab, which contains the execution path of the whole project or part of it. Basically, it captures everything the back-end engine is performing as it runs through each line of the code and puts it in an events file, which has the format – '.cme'. The middle section is the one where we write the code (the script file has the format - .tgp). There are two modes of coding, one is pure java, and the other is Intelli-mode, which is being shown here. We can switch between these modes by just right-clicking on the code section and choosing the other mode.

Presently, I cannot help with the installation part because I got the software already installed by my employer. But, with time, a situation may arrive where I may need to know.

Before we start coding with B2B DT, we need some basic knowledge about XML schemas and Regex expression. These are pretty simple, and you can easily find them on net. Well, this was just an introduction to the world of B2B. I will post soon about whatever I know. Hopefully, it helps both the readers and me.

For now, just remember the top level things such as - why B2B is used, how the screen looks, what data formats it handles etc.