biztalk enable tracking

BizTalk Server Best practices, Tips, and Tricks: #5 Tracking Data enabled

Welcome once again to another BizTalk Server Best practices, Tips, and Tricks blog post! In my previous blog post, I talked about some essential tips and tricks for BizTalk Server administrators:

And for BizTalk Server Developers:

Today, we are going back to the administration world and talking about another critical Best practice, Tips and Tricks for BizTalk Server Administrator: Tracking Data enable

biztalk enable tracking

#5 Tracking Data enable

There are several tracking settings available on BizTalk Server. The common ones may be Receive and Send Ports, Pipelines and Orchestrations, but we can also do some times of tracking in Business Rules and Schemas.

But normally, when we speak about Tracking Data, we refer to:

  • Message tracking: the process by which parts of a message instance, such as the message body, message properties, and metadata, are stored in a database, typically for debugging purposes and archival (nevertheless, I do prefer using another strategy for archiving).
    • Message instance parts that are tracked can subsequently be viewed by running queries from the Group Hub page in the BizTalk Server Administration console. Because we can view live data and past data (archiving), this can be a helpful tool for identifying and fixing problems in a development, staging, or production environment.
  • Process tracking: in other words, orchestration tracking or orchestration debugger, which tells us all steps that were executed inside a business process.

biztalk enable tracking

The tracking configuration settings allow you to track the following types of data:

  • Inbound and/or outbound event data. For example, message ID and start and stop times for the artifact.
  • Inbound and/or outbound message properties. For example, general and promoted properties for each message that the artifact processes.
  • Inbound and/or outbound message bodies and parts. For example, the body and parts for each message that the artifact processes.
  • Orchestrations. Execution data for orchestration shapes.

Data and event tracking

What is the main problem with Tracking Data?

One of the main problems with tracking is that normally Developers have their environments configured as Full tracking Enabled.  The reason why is that it’s easier to debug, troubleshoot, analyze or validate and see if everything is running well or simply what is happening with their new applications. This is ok for a developer environment. We need the tracking enabled to understand if our solutions are working as expected. We also need to monitor the tests, and this is the way to do it!

But this is NOT IDEAL in production environments. You should not have full tracking enabled by default in production – of course, we can enable full tracking for an application to troubleshoot production issues for a certain, and hopefully, a short period.

The important question is: Do Developers remember to disable tracking before they generate the MSI and binding for production?

No! And actually… to be honest, they don’t care about that! It is not their task to do it or control it. And if you ask them, they probably tell you that you should always have them enabled!

Asking or forcing developers to remember when they export an application to disable the Tracking data properties can be a big challenge… or even impossible! It will be an annoying and time-consuming “fight” that, in the end, you as administrator will lose. It will be the same as asking developers to change their way of being.

So what are your options as an administrator?

Before BizTalk Server 2016

If you are using a version older than BizTalk Server 2016, my advice is, if you are a BizTalk Administrator, let the developers be happy thinking they are annoying you and take back control of your environment by yourself.

These tasks can be easily automated and configured by easily creating or using PowerShell.

You should disable all Tracking or enable just the important settings at the application level. You may lose one day developing these scripts, but then you do not need to worry anymore about it.

As an example, with this script: BizTalk DevOps: How to Disable Tracking Settings in BizTalk Server Environment, you can easily disable all tracking settings for all the artifacts (orchestrations, schemas, send ports, receive ports, pipelines) in your BizTalk Server Environment

After BizTalk Server 2016

In previous versions of BizTalk Server, tracking settings were automatically imported with the rest of the application bindings. However, if you are working with BizTalk Server 2016 or 2020, you have a new feature that gives you better control while importing your BizTalk Applications: Import Tracking Settings.

If you are importing an MSI file, on the Application Settings tab, you will have a checkbox Import Tracking Settings that allows you to say: I don’t want to import the tracking from DEV or another environment in which the MSI was generated from.

Import Tracking Settings

If you are importing a Binding file, you will also have this same option:

biztalk enable tracking

Of course, if you want to properly define the correct or minimum tracking settings of your application, you need to do it manually or, once again, use a PowerShell script to accomplish that:

List of tracking options per artifact

Orchestrations

Option

Description

Track Events – Orchestration start and end

Select this check box to track the orchestration instance before and after processing the entire business process. Orchestration tracking lets you see the instances in the tracking query result views.

Track Events – Message send and receive

Select this check box to track message send and receive events. This check box is available only if you select the Orchestration start and end check box.

Track Events – Shape start and end

Select this check box when you need to debug orchestration instances in the Orchestration Debugger. When this check box is selected, the event list in the Orchestration Debugger is populated. This check box is available only if you select the Orchestration start and end check box.

Track Message Bodies – Before/After orchestration processing

Select this check box to save and track the actual message content prior/after to processing by the orchestration instance. This check box is available only if you select the Message send and receive check box

Track Message Properties – Incoming/Outgoing  messages

Select this check box to track the promoted properties of an inbound/outbound message.

Send Ports

Option

Description

Track Message Bodies – Request message before/after port processing

Select this check box to enable you to save and track request message content before/after the message is processed by the send port. 

Note: You must enable message body pipeline tracking to successfully track the response message before port processing

Track Message Bodies – Response message before/after port processing

Select this check box to enable you to save and track response message content before/after the message is received. 

Track Message Properties – Request message before/after port processing

Select this check box to track the promoted properties of an outbound message. before/after the message is processed by the send port. 

Track Message Properties – Response message before/after port processing

Select this check box to track the promoted properties of an inbounded response message before/after the message is processed by the send port. 

Receive Ports

Option

Description

Track Message Bodies – Request message before/after port processing

Select this check box to enable you to save and track request message content before/after the message is processed by the receive port. 

Note: You must enable message body pipeline tracking to successfully track the response message before port processing

Track Message Bodies – Response message before/after port processing

Select this check box to enable you to save and track response message content before/after the message is received. 

Track Message Properties – Request message before/after port processing

Select this check box to track the promoted properties of an inbound message. before/after the message is processed by the receive port. 

Track Message Properties – Response message before/after port processing

Select this check box to track the promoted properties of an outbound response message before/after the message is processed by the receive port. 

Pipelines

Feature

Description

Port start and end events

Select this check box to track only when an instance starts and ends. Details include item name, assembly, and other metadata.

Message send and receive events

Select this check box to track message send and receive events. This check box is available only if Port start and end events are selected.

Messages before pipeline processing

Select this check box to save and track the message bodies received by the pipeline, which holds metadata such as URLs and promoted properties. If this is a receive pipeline, the message body is the raw message as submitted to the pipeline by the transport component. Depending on the application, the message might be encrypted, signed, or encoded. When using a BizTalk map, if this is a receive pipeline, tracking takes place after the inbound map is processed.

This check box is available only if Message send and receive events are selected.

Messages after pipeline processing

Select this check box to save and track the message bodies sent by the pipeline, which holds metadata such as URLs and promoted properties. If this is a receive pipeline, the message body is the processed message to be submitted to the MessageBox database, which may be XML depending on your application. When using a BizTalk map, if this is a send pipeline, tracking takes place before the outbound map is processed.

This check box is available only if Message send and receive events are selected.

Schemas

Possibility of tracking all message properties

Business Rules Policy Execution

Option

Description

Fast activity

Select this check box to track the instance data on which the policy operates.

Condition evaluation

Select this check box to track the true/false results of conditions in the selected policy.

Rule firings

Select this check box to track the actions started as a result of the policy.

Agenda updates

Select this check box to track updates to the agenda. The agenda contains a list of actions that are “true” and need to fire.

Best Practices for Tracking Data

  • Determine the information you need to track during planning: You should decide during the planning stages which information you need to track so that after you deploy the project, you can set the tracking options and limit the amount of tracked data to give you only the information you need.
    • Of course, you should keep it to as minimum as possible, or you fill comfortable.
    • Enable more tracking when needed for debugging or troubleshooting, and when you have done, disable the unnecessary tracking settings.
  • Do not track all messages: We recommend that you not track all messages because each time a message is touched, BizTalk Server makes another copy.
    • You can instead narrow the scope by tracking only a specific port. This helps to maximize the performance of your system and to keep the databases uncluttered.
    • Once again, enable more tracking when needed for debugging or troubleshooting, and when you have done, disable the unnecessary tracking settings.
  • Set tracking on send ports and receive ports instead of on a pipeline: If you set tracking options on pipelines, you will also set the tracking options globally for every port that uses the pipeline.
    • This, in turn, may result in far more data being tracked than you intend, which will slow system performance. Instead, you can set tracking options on send ports and receive ports.
    • When necessary, enable tracking at the pipeline level for the minimum short period possible for debugging or troubleshooting.
  • Disable orchestration start and end shape tracking: This tracking is only useful for the orchestration debugger, and it can be used in production environments for debugging or troubleshooting a business process. However, this can have a large impact on the performance if there are a lot of long-running orchestrations. When this tracking is enabled, the events are saved in the “dta_debug_trace” table in the BizTalkDtaDb, it is recommended that this table doesn’t contain more than 1 million rows.

Stay tuned for the following BizTalk Server Best practices, Tips, and Tricks.