From time to time, we notice that BizTalk Server users struggle with understanding the concepts around the scalability of BizTalk Server. Therefore, we wanted to share some highlights in this blog on that topic. We will cover topics like the difference between load balancing and fail-over clustering and in which scenario to choose what technology from a BizTalk perspective. Next, we discuss several best practices for load balancing with BizTalk Server Host design.
Let’s first go back to the role of middleware software, continue with a brief explanation of the importance of scalability and look at the available options for BizTalk Server.
Middleware products like BizTalk Server enable you to integrate software products. A classic example is when you run a web shop like Amazon. Customers place orders in the web shop and to have those orders delivered, other software systems might be involved besides the web shop. Think of ERP systems, warehouse systems and logistic systems.
Using middleware products, order information, customer information, etc. can be shared automatically between the relevant systems. That saves much time because otherwise such information must be entered manually in all those systems. To have the order information etc. flow between the relevant systems, the middleware platform must be healthy and capable of processing all the incoming orders etc. This brings us to the topic of scalability.
Once you have installed your BizTalk Server environment, you might have only one server capable of picking up messages from a source system, processing them, and transmitting them to the destination system(s). BizTalk Server highly depends on SQL Server databases for purposes like storing the messages with their (intermediate) states, etc. The setup of the BizTalk Group could look like the picture below.
The above setup might work fine when you are not running a globally available webshopweb shop (like Amazon), but only some simple integrations of lesser importance when. However, if you are processing thousands of transactions (if not more) per day, and your business depends on a properly working middleware platform, the above setup might not do because of the limited processing power of only one BizTalk server and the potential risk of that server going down. In such scenarios,
it is important to be able to add more processing power to the environment, in other words, to scale the BizTalk environment.
When you want to add more processing power to the BizTalk Server environment and prevent the environment from going down due to the unavailability of the BizTalk (or SQL) server, concepts like load balancing and high availability come into the picture.
With BizTalk Server, multiple technologies can be used to make an environment load-balanced and highly available. Those technologies are:
For the rest of this blog, we give a short introduction to Windows Failover Clustering and Always On Availability Groups, but we focus on BizTalk Server Host Design.
Probably, the easiest and most used technology/concept that is used to make BizTalk Server load-balanced and highly available is by implementing a Host design. With a Host design, you create several Hosts that will be used for tasks like receiving and/or transmitting messages, processing orchestrations, and processing tracking information. Hosts can be assigned to BizTalk Receive Locations, Orchestrations, Send Ports, and Send Port Groups. Each Host has its own set of working tables in the BizTalk MessageBox database.
A Host can have one or more Host Instances (Windows NT Services) that run in the BizTalk servers. So, when you have three BizTalk servers in your environment, a Host can have three Host Instances: one Host Instance per BizTalk server. That way, you have three times more processing power and have achieved high availability because the Host Instances run on three different servers. Continue reading for best practices around BizTalk Server Host design.
The concept of Windows Failover Clustering can be used to make servers highly available in an Active/Passive configuration. Such a configuration looks like the below picture.
Concerning BizTalk Server, Windows Failover Clustering can be used for BizTalk Server itself, but also in the SQL Server data layer. In the case of clustering the SQL Server data layer, the cluster contains at least 2 SQL servers; one is Active and handles all the requests from BizTalk Server, and the other is Passive. BizTalk Server accesses the cluster via a virtual name, so it does not need to be aware of the currently active SQL server. If the active server (unexpectedly) goes down, a fail-over happens, and the passive SQL server becomes active and continues to handle the requests from BizTalk Server.
Windows Failover Clustering can also be used at the BizTalk Server layer. This is important when receiving messages/files via the POP3, MSMQ, or FTP adapters. In those cases, due to the nature of those protocols, transactionality cannot be guaranteed, forcing BizTalk to pick up messages with one Host Instance only. In this case, high availability can be achieved via a Failover cluster in an Active/Passive configuration. You can read more about this topic in the article Considerations for Running Adapter Handlers within a Clustered Host.
The Always On Availability Groups feature is a high-availability and disaster-recovery solution that provides an enterprise-level alternative to database mirroring. Introduced in SQL Server 2012 (11. x), Always On Availability Groups maximizes the availability of a set of user databases for an enterprise. An availability group supports a failover environment for a discrete set of user databases, known as availability databases, that fail over together. An availability group supports a set of read-write primary databases and one to eight sets of corresponding secondary databases. Optionally, secondary databases can be made available for read-only access and/or some backup operations.
An availability group fails over at the level of an availability replica. Failovers are not caused by database issues such as a database becoming suspect due to a loss of a data file, deletion of a database, or corruption of a transaction log.
Now that we have a basic understanding of the available load balancing and high availability options for BizTalk Server and in what scenarios they can be used, let’s discuss some best practices concerning BizTalk Server Host design.
As mentioned earlier in this article, a Host is used to segregate tasks like receiving messages, processing orchestrations, and transmitting messages. A Host can be seen as a logical container with Host Instances (Windows NT Services) that run in your BizTalk servers; each Host can have a maximum of one Host Instance in each BizTalk server in your BizTalk environment. The benefit of creating multiple Hosts (and Host Instances) is that it gives the administrator team the ability to manage and segregate workload.
A best practice for a Host Design is to create Hosts for the following tasks:
In the above image, the selected Hosts (in blue) are created for receiving/transmitting of messages, and processing orchestrations.
After you have created the receive and send Hosts, you should map them to the adapters you want to be used by those Hosts. Read more about that in the article What Is an Adapter Handler? on Microsoft Learn. When that is done, you can assign BizTalk ports and orchestrations to the hosts you created.
Besides creating Hosts for receiving, processing, and sending messages, you should also create a Host for processing tracked information. You could name that for example BizTalkServerTrackingHost. You must make sure that there is only one Host that has the Allow Host Tracking property enabled. For the BizTalkServerTrackingHost, you should create a Host Instance in each BizTalk server of the environment.
Although most of the adapters that come out of the box with BizTalk can run in 32-bits and 64-bits, there are also a couple of adapters that do not support 64-bit mode. Think of for example the FTP adapter and the POP3 adapter. If you are using those adapters, you could create a separate Host for those 32-bit adapters. You could name such a Host as BizTalkServerReceive32Host.
You might have an integration that has specific security requirements which are not required/allowed for the remaining integrations. In such cases, you could think of creating a separate Host and creating service accounts for the Host Instance(s) that match the requirements for that integration. You could think of assigning all the ports and orchestrations to that Host. Using such a separate Host makes the integration safe and easy to manage.
We have come to the end of this blog. If you want to manage and monitor your BizTalk Host Instances, monitor and view Host Throttling conditions, and view what all BizTalk360 has to offer, why don’t you check if the product would be good for you? You can download a free trial or request a demo to have an open conversation to understand how the product solves your monitoring and administration challenges.