RabbitMQ is an open-source tool used as a message broker to transmit messages between the two distributed systems. RabbitMQ uses one of the advanced messaging protocols (AMQP), which is the source application layer protocol for middleware.
The architecture of a message queue is simple; there are client applications called producers that create messages and deliver them to the queue. RabbitMQ uses a publish-subscription model to transmit the messages between the source application and the destination application. The Queue manager acts as a middleman to process the published messages to subscribers’ applications.
The producer sends messages to an exchange instead of queuing directly. In turn, an exchange is routing the messages to different queues with the help of bindings and routing keys.
In a single RabbitMQ instance, Vhost /virtual host segregates the applications. The users can have permissions for different hosts, queues, and exchanges.
RabbitMQ has a web-based management console used to create and manage:
Microsoft BizTalk Server is a middleware platform that supports Windows MSMQ for message brokering. It supports other queuing mechanisms like IBM MQ, Azure Service Bus Queues, and RabbitMQ.
In the recent past, most organizations have used hybrid solutions which use advanced messaging protocols. Integrating RabbitMQ with BizTalk enables users to leverage the queuing features for building robust and fault-tolerant integration solutions.
The RabbitMQ Bindings are not blown-out-of-box solutions from the BizTalk Server. RabbitMQ works only with a WCF-Custom adapter. Configure the RabbitMQ Binding properties as in this article, which transports the messages using the hostname, Virtual Host, and user credentials properties. This custom adapter is used in Receive Locations and Send Ports.
The queues allow the web servers to respond to requests spontaneously instead of being forced to perform resource-heavy procedures to avoid delayed response time. Message queueing is important when you want to distribute a message to multiple consumers or to balance loads between workers.
The users who are using RabbitMQ queues have challenges in keeping track of the health and processing status of RabbitMQ queues. BizTalk360 extended its support to monitor RabbitMQ in version 10.9, which helps with real-time issue identification, including high queue lengths, message delivery failures, or slow consumer processing. This method involves a two-step process
Most of the organizations have hosted RabbitMQ Server in a standalone machine in the BizTalk environment. To monitor the RabbitMQ Queues, the user must configure in the environment settings → Monitoring → Mapping configuration.
RabbitMQ host configuration requires a host server name, port number, and user credentials to connect the RabbitMQ Host and its container queues.
The users can add multiple hosts for monitoring based on the business vertical and architectural design.
While configuring the RabbitMQ Host, the user can choose the Virtual Hosts they want to monitor the queues. As the default configuration, all the Virtual Hosts in the RabbitMQ Server are selected.
Orphaned Queue: When the “Deleted Orphaned Queue” option is enabled in Host configuration if the mapped queue for the monitoring is deleted from the RabbitMQ Server, the configuration of the queue is automatically removed from the RabbitMQ mapping.
To monitor the RabbitMQ queues, configure the mapping in the Monitoring → Manage Mapping → Queues → RabbitMQ.
Manage the mapping for the monitoring by the Host configured in BizTalk360 configured. In the host configuration, users can choose multiple virtual hosts for monitoring.
After establishing a successful connection with the configured RabbitMQ hosts, the system can retrieve information about all the available queues linked to these hosts and present them in the RabbitMQ Monitoring section. From here you can monitor the queue by setting the expected state or by configuring the threshold rule.
Mapping the queue’s expected states helps to determine the status of the RabbitMQ queues. The monitoring states of queues are:
There are different types of queues available, and their default state differs for each type of queue.
When the expected state matches the current state of the queue means it’s in a healthy state. If the state is not in the expected state, then it’s in a critical state. When an Integration Queue is the type of Quorum whose expected state is set to running (default state).
Similarly, for the classic queue (Invoice Process), the state is always Idle; when the expected state is set as running, it means it’s in a critical state. According to the type of queues, set their expected state of monitoring.
RabbitMQ takes care of the message transmission between two distributed systems. To keep track of threshold violations in the message processing with the different metrics are
The finance application handles the multiple vendors’ document processes across the different applications. Message exchanges happen between applications through RabbitMQ queues.
The operator users/admins must manually keep track of message flows between the applications. Identifying causes leads to bottlenecks like lack of performance or a large volume of messages processed in a queue.
In this case, rules monitoring of the RabbitMQ queue helps to identify threshold violations of metrics like messages persistent or the size of messages persistent in the queue. This threshold violation indicates the performance hit in the message processing.
When the number of active messages is more than 10, it leads to a throttled situation. The users must take care of message loading in the queues when large documents are processed. Administrators can investigate the infrastructure to improve the performance of message processing.
Note: Document processing happened via Stream Queue Type.
The monitoring dashboard reflects the status of the RabbitMQ Queues. When a threshold violation occurs, the notification is pushed via email and notification channels.
The Invoice process queue must deliver the invoice messages to the consumer applications. In return, the vendor application accepts the acknowledgment of the invoice receipt.
Condition #1: If the Messages Unacknowledged Rate is Greater than or Equals 50, the BizTalk360 monitoring service alerts about the warning condition.
Condition #2: The Invoice queue must process the message to three consumers(applications); if monitoring conditions fail, the monitoring services push the critical notifications
This article provides insights into the RabbitMQ Queue’s state monitoring and message processing threshold violations. This integration helps the users and extends its support to monitor the RabbitMQ Queues in BizTalk360.
