BizTalk Pipeline

Archive Pipeline Component for BizTalk Server

There is a new Custom pipeline available on my BizTalk Pipeline Components Extensions Utility Pack project, that is available on GitHub. For those who aren’t familiar with it, this project is a set of custom pipeline components (libraries) with several custom pipeline components that can be used in receive and sent pipelines. Those pipeline components provide extensions of BizTalk’s out-of-the-box pipeline capabilities.

This month my team and I updated this project with a brand new component: BizTalk Archive Pipeline Component.

BizTalk Archive Pipeline Component

The BizTalk Archive Pipeline Component is based on the initial work of Randy Paulo‘s BizTalk Archiving SQL and File component that my team and I adjusted and migrated to BizTalk Server 2020. This component was initially available on CodePlex which is unreachable since it was closed.

The component enables you to archive incoming/outgoing messages from any adapters to either a folder (local, shared, network) or in a SQL Server database. 

The project also includes a SQL script file called CreateDatabase.sql that you can use to create the supported resources in SQL Server to be used with this component:

  • Database called BizTalkArchiveDb with a table named Messages
  • Stored procedure called InsMessages

This is the list of properties that you can set up on the archive pipeline component:


Property Name

Description

Use for

Sample Values

Notes

CompressionPassword

The password that will be used in the zip file

File & SQL

P@ssw0rd

 

DbConnProvider

Connection Provider

SQL

System.Data.SqlClient

 

DbConnStr

Database connection string

SQL 

Data Source=localhost;Initial Catalog=BizTalkArchiveDb;Integrated Security=SSPI;

Using integrated security, you need to grant the identity of the host wherein the receive/send port is bound permission to BizTalkArchiveDb with ff roles: dbdatareaderdatawriter, db and execute permission on the stored procedure InsMessages

DbFileExtensions

File extension to be used

SQL

.xml

 

DbPropList

This is a set of delimited values of name & namespace of message context properties that will be passed to the stored procedure to be used for custom logging.

SQL

ReceivedFileName;http://schemas.microsoft.com/BizTalk/2003/file-properties|ReceivedPortName;http://schemas.microsoft.com/BizTalk/2003/system-properties

The format is Property Name;Namespace and for multiple values, the delimiter is ‘|’ pipe symbol.

DbSPName

Stored procedure name

SQL

Built-In: InsMessages

The stored procedure can be customized as long as it has the same signature as the default one. For instance, there can be additional logic to parse the Message Properties and insert them into the new custom table

FileArchiveBackFolder

Folder / Shared Location / Network location wherein the files will be archived

File

\\archive\Test\Archive

If it’s a shared folder, the pipeline will try to use first the identity of the host. If it failed, it will use the credentials supplied (see FileArchive UserName, Password, Domain)

FileArchiveFileName

Name of the archive file

File

%SourceFileName%_%datetime%

Supported macros are: – %SourceFileName%, %MessageID%,%datetime%, %time%

FileArchiveIsOverwriteFiles

Overwrite Flag

File Archiving

True

 

FileArchiveUserDomain

The domain name of user credentials

File Archiving

DOMAIN

This value will be used if the identity of the host instance doesn’t have permission on the backup folder.

FileArchiveUserName

User name

File Archiving

BizTalkUser

This value will be used if the identity of the host instance doesn’t have permission on the backup folder.

FileArchiveUserPwd

Password

File

P@ssword

This value will be used if the identity of the host instance doesn’t have permission on the backup folder.

IsArchiveToDb

Archive to database flag

SQL

True

True = Archive to database, False = Disabled

IsArchiveToFile

Archive to file flag

File

True

True = Archive to file,
False = Disabled

IsCompressFile

Compress file flag

SQL & File

True

True = Compress the file, False = No compression, It uses DotNetZip

How to install it

As always, you just need to add these DLLs on the Pipeline Components folder that in BizTalk Server 2020 is by default:

  • C:\Program Files (x86)\Microsoft BizTalk Server\Pipeline Components

In this particular component, we need to have these 3 DLLs:

  • BizTalk.Archiving.Common.Configuration.dll
  • BizTalk.Archiving.PipelineComponents.dll
  • Ionic.Zip.dll

If you prefer, you can instead add all of these 3 DLLs in the GAC or leave the BizTalk.Archiving.PipelineComponents.dll on the pipeline component folder and add the other 2 in the GAC. 

As you see you have several options.

How to use it

To use the pipeline component, I recommend you to create a generic or several generic pipelines that can be reused by all your applications and add the BizTalk Archive Pipeline Component in the stage you desire. The component can be used in a stage of the receive and send pipelines.

Deploy the pipeline to your environment and configure it accordingly.

Download

THIS COMPONENT IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND.

You can download the BizTalk Archive Pipeline Component from GitHub: