Azure Service Bus Monitoring

Monitor Azure Service Bus Topics and Subscriptions in BizTalk360

Introduction

Hybrid – A word that has gained increasing attention in recent years. The term “hybrid” refers to a combination of more than one entity (most of the time 2). For instance, a hybrid vehicle that runs on both electricity(battery) and gasoline(petrol). Similarly, the Hybrid Integration Platform (HIP) integrates both on-premise and cloud-based systems. The HIP is becoming a predominant chapter in most of the firms’ integration journey.

Microsoft BizTalk Server is often described as the “middleman” in integration that helps to connect various systems and applications together. Apart from being the best-suited integration server for on-premises integration platforms as well as Infrastructure as a Service (IaaS), Microsoft BizTalk Server is also integrating cloud services with a variety of built-in adapters for the cloud, such as Azure Logic Apps, Service Bus, Event Hubs, Azure Blob Storage, API Management and So on. This makes applications and services that utilize on-premises systems but still leverage cloud that provides greater connectivity and elasticity.

Among these numerous flavors of options to facilitate hybrid integration, in this blog, let’s look at connecting Microsoft BizTalk Server with Azure Service Bus, specifically Service Bus Topics, using the SB-Messaging Adapter, and how BizTalk360 can help you in effectively monitoring the same.

What is the SB-Messaging adapter?

The SB-Messaging adapter in BizTalk Server is used to easily connect with Azure Service Bus entities and provide the ability to do reliable messaging. The SB-Messaging adapter can send and receive messages from Service Bus entities such as queues, topics, and subscriptions.

Use-case

SB-Messaging adapter

This illustration depicts one of the scenarios in the hybrid integration of on-premises BizTalk Server with Azure Service Bus. In this context, Organization “A” sends a large number of messages to the Azure service bus topics from a system running on-premises using a send port with the SB-Messaging transport type. Those messages will be delivered to the various subscriptions available under the topic. Messages can be received by several other organizations by connecting to the subscriptions using a receive location with the SB-Messaging transport type.

Connectivity Setup

For both sending and receiving messages to and from the service bus, we only need the Service Bus entity URL and Access Control Service (ACS) or Shared Access Signature (SAS) credentials.

Let’s take a look at how simple it is to send messages to a service bus topic.

  1. Create a one-way send port and set the transport type to SB-Messaging.
  2. Connectivity Setup
  3. Click “Configure” to configure the transport properties of the send adapter. In the Destination URL enter the namespace of the service bus in the appropriate place and the topic name at the end of the URL.
  4. sb://.servicebus.windows.net//

    SB Messaging
  5. Copy the connection string from the Azure portal under the Shared access policies section. Go to the Authentication tab. Enter the shared access key name and access key from the connection string.
  6. Authentication
  7. In the Properties tab, add a label to identify messages in the Service Bus/BizTalk. Specify the other properties of the Send port like filters, pipeline, handler, etc.
  8. Properties
  9. Click “OK” to finish creating the Send port. Now enlist and start the send port. Outgoing BizTalk messages will now be posted to the Service Bus topic by the adapter.

Similarly, for receiving messages from a specific service bus Subscription, create a Receive Location with SB-Messaging as the Transport Type. While configuring, specify the Destination URL as follow:

sb://<Namespace>.servicebus.windows.net/<TopicName>/Subscriptions/<SubscriptionName>

General

You can also set connection properties like open/close/receive timeout and the prefetch count. The prefetch count determines how many messages BizTalk Server will fetch at once.  This can also aid in increasing the adapter’s throughput. After entering the SAS key name and key-value from the connection string, click Ok. The BizTalk Server will now begin retrieving messages from the Subscription. As Easy as PieūüėÄ

In real-time, a complex integration journey does not end after connecting the components and exchanging messages; it is vital to keep on monitoring the components and sending automated alerts whenever something goes wrong. This ensures that blind spots are eliminated and that “healthy” benchmarks are established.

What could possibly go wrong?

Indeed, there could be a lot of reasons! But to be more specific to the above scenario we can mention few issues,

  • BizTalk Server Artifacts or Service Bus entities may become unavailable at times or be accidentally disabled. If any one of the components fails, the entire integration process will be adversely affected
  • Service bus subscriptions may be bombarded with an excessive number of active messages, potentially exceeding the limit, and increasing the load
  • When messages cannot be processed or delivered to any receiver, they are routed to the dead-letter queue and remain there until someone investigates

In such scenarios, it’s important to keep an eye on both the BizTalk Server platform and the Service Bus Topic and Subscriptions to which it is connected to ensure that everything is running smoothly and without downtime.

Fortunately, BizTalk360 provides almost everything that we need to proactively monitor Microsoft BizTalk Environments and receive up-to-date alerts and notifications. To keep up with the Integration trend, various Azure Services such as Logic Apps, API Apps, API Endpoints, and Service Bus Queues have also been monitored in BizTalk360. We are super happy to extend our support to monitor Azure Service Bus Topics in the upcoming BizTalk360 v10.1. This blog gives you a clear picture of how we can monitor Azure Service Bus Topics and Subscriptions with BizTalk360.

Topics and Subscriptions Monitoring 

In BizTalk360 v10.1, after creating an alarm, one can start monitoring a topic and its subscriptions by simply entering the Namespace Connection String and the Topic name.

Topics and Subscriptions Monitoring

BizTalk360 will then automatically list the details such as the topic’s current state, size, number of subscriptions, and their details.

Service Bus Topics and Subscriptions

State based monitoring for service bus topic

To monitor whether the configured topics are in the expected state, you can set the “Excepted State” for a single topic or set of topics to one of the three states listed below:

  • Active (The topic is active. You can send messages to the topic)
  • Disabled (The topic is suspended. You can’t send messages to the topic)
  • Send Disabled (The same effect as Disabled. You can’t send messages to the topic. You’ll get an exception if you try to send messages to the topic)

Whenever the topic’s current state and expected state mismatch, BizTalk360 will send you an alert to notify the same. Similar to the auto-healing option available in other features, here also you can monitor the topic state without any manual intervention. When the auto-correct option is enabled for a topic with a maximum retry count and a reset interval, the monitoring service will attempt to restore the topic’s state to the specified expected state.

Alert rules for various key metrics

In addition to state monitoring, you can also set threshold rules for Topic Size and Count of Scheduled messages present in the topic. When defining the threshold criteria for Topic size, the threshold value can be specified in any of the following units: Bytes, KB, MB, or GB. Then you can also see the actual size of the topic in the specified unit.

Alert rules

In case of Subscriptions, you can set threshold rules for the below-mentioned metrics,

  • Subscription State (Current Status of the subscription)
  • Message Count (Total number of messages in the subscription)
  • Active Message Count (Number of messages in the subscription that are in the active state and ready for delivery)
  • Dead letter Message Count (Number of messages in the dead-letter queue)
  • Transfer Message Count (Number of messages pending transfer into another queue or topic)
  • Transfer Dead letter Message Count (Number of messages that failed to transfer into another queue or topic and have been moved into the transfer dead-letter queue)
  • Scheduled Message Count (Number of messages in the scheduled state)

For example, consider the following scenario:

We want to make sure that a Subscription state is in Active State so that they can receive the messages and consumers can consume them. Also, we want to check that if the active message count is greater than 1000 since this can indicate an issue with the service’s capacity to keep up with the load that is being placed on it. Furthermore, we want to check there are no dead-letter messages at a subscription since that indicates the message(s) haven’t yet reached the intended recipient and should be inspected. As a result, in such cases, you can set a threshold rule as below to trigger an alert.

Azure Service Bus subscription

As shown in the above examples, you may combine multiple check conditions using logical AND/OR operators.

Once the threshold rules are configured, all the rules will be evaluated, and the monitoring status of the topics and subscriptions will be updated. You can quickly view the summary of all rules of a particular topic or subscription by clicking on the “eye icon” at the end of the view.

Monitor results

If the actual values of any Topic or Subscription violate the specified rule at any point in time, an alert notification will be sent to the mapped email/contact. In the generated email, you can see the overall monitor status of all configured topics and subscriptions, the monitor status of a specific topic, and the details of state or threshold violation. Under a topic, you can see the monitor status of each subscription as well as a brief description of the violated rules.

Here is a sample picture of an email alert

email alert

Note: We have improved our default email template style to provide better readability for Topics and Subscription related alerts. If you are upgrading from an earlier version of BizTalk360 to v10.1, you can either create a new template and map it to an alarm or you can restore the existing template to the updated one.

Conclusion

We hope that this new feature will make our customers’ hybrid integration scenarios go more smoothly.

Do you want to manage and monitor your Microsoft BizTalk Environments and Azure Services in the best possible way? Click here to start your 30-day free trial and check out more exciting features.