To be able to analyze any issues during the processing of messages in the BizTalk Server, it helps to understand how messages are being processed through your BizTalk environment. For example, think of relevant information like wherein the BizTalk has a message been picked up, which pipelines and transformations have been used, which orchestrations have been involved, and what send port(s) transmitted the message outbound.
In many cases, you will be able to start an analysis of issues by running MessageBox queries from the Administration console (or in BizTalk360). However, what should you do when a transaction has been completed and cannot be found anymore in the MessageBox? Information about completed transactions might be found via Tracking queries. However, to have that visibility Tracking needs to be configured upfront. In this blog, we explain BizTalk Message Tracking in a bit more detail and provide a couple of best practices.
The importance of BizTalk Tracking
Integration platforms often form the spider in the web in the information landscape of an organization. With BizTalk Server, Microsoft’s on-premise integration platform, it is just the same. It is used to connect on-premise systems, but can also be used to integrate, via the internet, with branch offices, partners, and SaaS solutions. When organizations have a somewhat bigger footprint in BizTalk Server, thousands (likely even many more) of transactions are processed by the platform. It will be clear that with such huge amounts of transactions sometimes things go wrong, resulting in business transactions not being processed as you would expect. The nature of such exceptions is diverse and can include both technical as well as functional root causes.
What can be tracked with BizTalk Server
To identify error situations, middleware platforms often allow you to track specific information about the processed transactions. In the case of the BizTalk Server, you can track the following:
- Event Tracking – Track when a message was picked up from an upstream system when it was processed by orchestrations, and/or when it was delivered to downstream systems
- Content Tracking – Track the actual message bodies (content) and information about the messages (context)
To be able to track events and/or content in BizTalk Server, it is required to configure specific components in BizTalk Server. Such components are:
- Receive Ports
- Send Ports
How to configure BizTalk tracking
Now we have seen what kind of information can be tracked, let us open the BizTalk Server Administration Console and find out where the Tracking settings can be found. However, we shortly have a look at another very important tracking-related setting.
Enable group-level tracking
Before we move on which the tracking settings at the BizTalk application level, let’s shortly discuss enabling tracking at the group level. The BizTalk Settings Dashboard contains a setting called ‘Enable group-level tracking’. It is important to realize that when this setting is disabled no tracking at all will be done. However, even when you don’t want to perform any tracking, it is advised to leave this setting to be enabled. This BizTalk Settings Dashboard can be found by right-clicking on the BizTalk Group and select Settings from the context menu. Check the screenshot above to find the BizTalk Group. You can read more about the BizTalk Settings Dashboard in this article.
Enabling BizTalk Application-level tracking
The below screenshot shows the BizTalk Server Administration Console with the BizTalk EDI Application already expanded. The arrows point out the artifact categories for which tracking can be enabled/disabled.
We will open a couple of artifacts and show you the tracking settings.
Receive Port tracking
First, let us check the tracking settings of a Receive Port.
Here, we have opened a one-way Receive Port. The Tracking page of the port is already showing up. As you can see, you can track the following:
- Message Bodies
- Request message before port processing – Track the message body how it has been picked from the endpoint, before the execution of the pipeline and map
- Request message after port processing – Track the message body how it looks like after execution of the pipeline and map
- Message Properties
- Request message before port processing – Track the message properties how it has been picked from the endpoint, before the execution of the pipeline and map
- Request message after port processing – Track the message properties how it looks like after execution of the pipeline and map
In this Receive Port, the message bodies are tracked before and after port processing. Note, that at this level content tracking is done, in the Receive Port it is not possible to perform event tracking! Event tracking can be done in pipelines.
When we would have opened a two-way Receive Port, that accommodates Receive Locations that connect to webservices, we would find tracking settings for Requests and Responses. This enables you to track both outgoing requests to a webservice, but also the incoming response from the webservice.
Note: Incoming EDI, HL7, or any other non-XML messages can be viewed in their native form by tracking the messages in the Receive Port before pipeline processing. This can be very beneficial because the XML translation, in the pipeline, can go wrong. If messages are translated to non-XML formats in the Send Port, you can track them by enabling tracking after port processing.
Next, let us have a look at the tracking settings of an orchestration.
Orchestration tracking settings contain the same settings as we have seen in Receive Ports, but on top of that, you can also track the following events:
- Orchestration start and end – Date and time when orchestration instances are started and ended
- Message send and receive – Date and time when orchestration instances receive and send messages
- Shape start and end – Date and time when shapes in orchestration instances are started and ended
The ability to track the start and end of shapes can be handy to find out which shapes are all executed for an orchestration instance, but also when an error has occurred in a specific shape. This can be very useful information when analyzing an issue.
If you want to track events about when messages are received by a Receive Port or transmitted by a Send Port, you need to use Pipeline tracking. The below screenshot shows the tracking configuration of the XMLReceive pipeline. This is one of the default pipelines that come with the BizTalk Server. It can be found in the BizTalk.System application.
As you can see, besides event tracking, pipeline tracking also enables you to track message bodies before and after pipeline processing.
Note: Be aware that when you turn on/off tracking on pipelines, that it will impact all the ports that are using these pipelines. Especially with standard pipelines like XMLReceive and XMLTransmit that might result in a bigger impact than you had in mind.
Challenges and guidelines with BizTalk tracking
The ability to track message events and content is very powerful in the day-to-day work of BizTalk Server administrators, as it enables them to understand how messages have been processed. However, we also often see people struggling with BizTalk tracking because they are afraid of any performance penalty.
In other words, the main challenges around using BizTalk Message tracking are:
- Risk of performance decrease because too much tracking is turned on
- No visibility when tracking is disabled because not enough tracking is turned on
Bottom line is that there must be a balance in the amount of tracking settings that have been turned on. There are a couple of guidelines that can be of help. Think of:
- Keep a good overview of what tracking settings have been turned on – By keeping the overview you prevent that more tracking is turned on than absolutely required. That will limit the impact that tracking potentially(!) has on the performance of your BizTalk environment. For example, you could temporarily turn on tracking in case of any issues with a specific integration, but turn it off when everything works correctly
- Be careful with the amount of message content tracking that has been turned on – In contrast with Event tracking, Message Content Tracking can be expensive, because the tracked messages might be big (think of batch messages of multiple Megabytes). However, tracking small messages (a couple of Kilobytes) should not harm the environment. In other words, try to be a bit careful with turning on Message Content in case of large messages
How does BizTalk360 help with BizTalk tracking
BizTalk360, being a One-stop product for monitoring and operating BizTalk Server environments has multiple features around BizTalk Server tracking. These features are:
- Graphical Flow (Tracking)
- Tracking Manager
We want that users of BizTalk360 are more productive. That is why we are always trying to bring smart features into the product that help our users to do their job as efficiently as possible. Also, with respect to the above-mentioned features, we have brought a couple of improvements compared to the BizTalk Server Administration console. Therefore, let us give you a short overview of both features.
What does the BizTalk360 Graphical Flow (Tracking) solve
This feature has similar capabilities like the Administration Console to query tracked messages and tracked service instances. You can create queries and apply filters to find the instances you are looking for. However, in the Administration console, it is hard and time-consuming to correlate messages that are processed by multiple ports and orchestrations. In other words, it is hard to discover the complete flow of a specific message.
BizTalk360 addresses that issue by providing a graphical representation of a specific tracked message. Besides that, by the click of a button, you can easily understand how the message has progressed through the BizTalk environment. See the below animation to experience the Graphical Flow in BizTalk360.
You can read more about the Graphical Flow (Tracking) feature in the Documentation Portal.
What does the BizTalk360 Tracking Manager solve?
We already mentioned the importance to keep an overview of what tracking settings are turned on or turned off. To have that overview, that means that in the Administration console you need to open all the ports where tracking settings can be found. That is not very efficient and time-consuming.
That is why BizTalk360 has the Tracking Manager which gives a consolidated overview of all the tracking settings. Besides the ability to view all the tracking settings, you can also configure the tracking settings. With the Tracking Manager in BizTalk360, you will better be able to have an overview of what is tracked and what is not tracked. Check to below animation to see the Tracking Manager in action.
You can read more about the Tracking Manager in the Documentation Portal of the product.
In this article, we have given you an overview of BizTalk Message Tracking and provided you with some guidelines about the usage of tracking in the BizTalk Server. On top of that, we have also shortly explained how BizTalk360 can help you to be more productive and have better control over the tracking settings.
Has this article raised your interest in working more efficiently and have better control over your BizTalk environment? Why not give it a try! We are happy to give you an obligation-free demonstration or you can take BizTalk360 to the trial!