Welcome once again to another BizTalk Server Best practices, Tips, and Tricks blog post! In my previous blog post, I talked about some essential tips and tricks for BizTalk Server administrators:
And for BizTalk Server Developers:
Today I’m going to speak about another critical Best practice, Tips and Tricks for BizTalk Server administrators and developers: Maintaining Archiving Folders.
This is a task that is the responsibility of administrators as well as developers! For sure, administrators need to implement some monitoring and cleaning operations, but developers need to report and help administrators identify all of these folders.
It is very common in integration scenarios to see messages being archived locally into a hard drive – either by using a pipeline component like:
or by simply using the default functionalities in BizTalk Server, like Send Port Filters:
This can happen for several reasons, for example:
The common problem with some of these approaches is that we usually tend to implement the archiving mechanism and we forget to implement a monitoring and cleaning mechanism. We end up with several problems after a while:
It is very easy to understand that if you have an integration process that processes hundreds of messages, you can quickly get this kind of problem.
Needless to say that it is extremely important that you implement monitoring and maintenance tasks from day one… or once again, you will completely forget until it is too late.
If you have monitoring tools like BizTalk360, you can take advantage of the existing monitoring capabilities that this tool will provide to you. One of that is the capability to monitor local or remote folders like share folders, FTP or SFTP. You can check more about this feature here: Introduction to File Locations Monitoring in BizTalk360, but in resume, you can set up a local folder – in this case, we will use a local folder – and configure a Thresholds rule with the metric File Count to monitor like:
You can also use a separate PowerShell script, and here we basically can do everything we want. We can monitor any folder we want and notify the teams if we want. But most importantly, we can implement cleaning tasks or maintenance tasks to avoid performance problems or prevent the disks from being full. To run this script, you could consider Automated Tasks in BizTalk360.
For example, this script:
|
4
5
6
7
8
9
10
11
12
13
14
15
16
17
|
foreach ($directory in $Directories){ $zipFile = "E:\BackupFileWarehouse\" + $directory.FullName.Substring($directory.FullName.LastIndexOf(":")+2).Replace("\","_") + "_" + (get-date -f yyyyMMdd_hhss).ToString() + ".zip" $filelist = get-childitem $directory.FullName | where-object {$_.LastWriteTime -le (get-date).AddHours(-$Hours)} | where-object {-not $_.PSIsContainer} if($filelist.Count -gt 0) { $filelist | format-table -hideTableHeaders FullName | out-file -encoding utf8 -filepath lastmonthsfiles.txt & 'C:\Program Files (x86)\7-Zip\7z.exe' a $zipFile `@lastmonthsfiles.txt $filelist | %{Remove-Item $_.FullName } rm lastmonthsfiles.txt }} |
THIS POWERSHELL IS PROVIDED “AS IS” WITHOUT WARRANTY OF ANY KIND.
Allows you to:
Of course, if this script will not move/create the archived files in a shared location or something similar, then you will need an additional mechanism to maintain and delete these zip files. This is just an example of how simple we can do these tasks.
You can download Manage messages being archived locally into the hard drive with PowerShell from GitHub here: Manage messages being archived locally into the hard drive with PowerShell
