Azure Blob Storage Archive Pipeline Component for BizTalk Server

Another message archive pipeline component is available on my BizTalk Pipeline Components Extensions Utility Pack project on GitHub! Yes, you read correctly. I think to finalize my adventure in archiving pipeline components for BizTalk Server. I had to create some component for the hybrid story, and the missing piece could be anything other than being able to archive all the messages on an Azure Blob storage account.

Azure Blob Storage is massively scalable and secure object storage for cloud-native workloads, archives, data lakes, high-performance computing, and machine learning, but it can also work as a perfect scenario to archive messages from BizTalk Server. This way, we don’t need to worry about disk space on our machines.

Once again, for those who aren’t familiar with it, the BizTalk Pipeline Components Extensions Utility Pack project is a set of custom pipeline components (libraries) with several custom pipeline components that can be used in receive and sent pipelines. Those pipeline components provide extensions of BizTalk’s out-of-the-box pipeline capabilities.

Today we will be adding a brand new pipeline component that you can use in all stages of your Receive and Send pipelines: the Azure Blob Storage Archive Pipeline Component. 

Azure Blob Storage Archive Pipeline Component

Azure Blob Storage Archive Pipeline Component is, as the name suggests, a component that enables you to archive incoming/outgoing messages from any adapters to an Azure Blob Storage account. 

This is the list of properties that you can set up in the archive pipeline component:

How to use it

Of course, to be able to use it we need to create a Storage Account in our Azure Subscription. To do that you need to:

• Log into your Azure Subscription at https://portal.azure.com
• Click New, from the categories select Storage, and then select Storage Account

• On the Create a storage account window, select:
  • The Subscription and the Resource group where you want to create the storage account – you can always create a new resource group.
  • Provide a storage account name, something like: striteaibiztalkpoc (template: str{bu}{dept}{process}{env}) 
    • The name must be unique across all existing storage account names in Azure. It must be 3 to 24 characters long and can contain only lowercase letters and numbers.
  • Choose the Azure Region that’s right for you and your storage account.
    • Not all storage account configurations are available in all regions.
  • In the Performance section, determine whether you want to have premium or standard performance for block blobs, file shares, or page blobs in your storage account.
  • In the Redundancy section, choose a replication strategy that matches your durability requirements.
    • The data in your Azure storage account is always replicated to ensure durability and high availability. 
    • Some settings can’t be changed after the storage account is created.
• Finally, click Review + Create. You can always customize even more by going to the Advanced setting and so on.

• Wait a few moments for your storage account to be created. Once completed, we need to access our brand new Azure storage account.
• In the left menu for the storage account, scroll to the Data storage section, then select Containers.
• Select the + Container button.
• Type a name for your new container, like biztalk-app-storage
  • The container name must be lowercase, must start with a letter or number, and can include only letters, numbers, and the dash (-) character. 
• Set the level of public access to the container. The default level is Private (no anonymous access).
• And then select Create to create the container.

Now we have everything we need on Azure. To use the pipeline component on our BizTalk Server environment, I recommend you to create a generic or several generic pipelines that can be reused by all your applications, and add the Azure Blob Storage Archive Pipeline Component in the stage you desire. The component can be used in a stage of the receive and send pipelines.

Deploy the pipeline to your environment and configure it accordingly.

How to install it

As always, you just need to add these DLLs in the Pipeline Components folder that, in Microsoft BizTalk Server 2020, is by default:

• C:\Program Files (x86)\Microsoft BizTalk Server\Pipeline Components

In this particular component, we need to have these  DLLs:

• BizTalk.PipelineComponent.BlobArchive.dll
• Microsoft.Azure.KeyVault.Core.dll
• Microsoft.Bcl.AsyncInterfaces.dll
• Microsoft.WindowsAzure.Storage.dll
• System.Memory.dll
• System.Numerics.Vectors.dll
• System.Runtime.CompilerServices.Unsafe.dll
• System.Threading.Tasks.Extensions.dll
• System.Diagnostics.DiagnosticSource.dll
• Microsoft.Azure.KeyVault.Core.dll

Download

THIS COMPONENT IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND.

You can download the Azure Blob Storage Archive Pipeline Component from GitHub:

• Azure Blob Storage Archive Pipeline Component