Large files routing approach in BizTalk

Problem: Large files routing in BizTalk

 Approach:

1) Create a WCF LOB Adapter/extend the SDK File adapter to implement file system watcher to monitor for files in a folder.

2) Once the file is created in the folder, create a new message containing the full path of the file and drop into message box via a custom receive pipeline component which promotes the absolute path of the file (c:\In\LargeFileName.xyz)

3) Create a custom send pipeline component which creates a new message and we associate the messageBody  property to the file (available in the context) using file stream (c:\In\LargeFileName.xyz)

4) Create a send port with the message type filter subscribing to the message created in step 2 and use the send pipeline component created in step 3.

This will also support retries and we have completely  eliminated MsgBox which would be a bottleneck

Using EntLib with BizTalk.

1) Create a new c# project and add all the necessary entlib dlls that you need. (this will help because we would not be adding references to entlib dlls for every biztalk project.

2) Create static wrappers classes for say Exception handling

  Policy- is the name that you give when you configure the Exception handling block.

public static class Helpers
    {

        public static void LogException(string policy, Exception ex)
        {
            ExceptionPolicy.HandleException(ex, policy);
        }

    }

3) Add  a reference to the above project in the biztalk project and invoke it as if you are invoking any other .net method

4) Create app.config file using the EntLib tool say BizTalkEnlib.config

5) Modify the BTSNTSvc.exe.config to include the following section right under the configuration node
 <configSections>
    <section name=”enterpriseLibrary.ConfigurationSource” type=”Microsoft.Practices.EnterpriseLibrary.Common.Configuration.ConfigurationSourceSection, Microsoft.Practices.EnterpriseLibrary.Common, Version=4.1.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35″ />
  </configSections>

6)Include the following right before the close of the configuration node

 <enterpriseLibrary.ConfigurationSource selectedSource=”File Configuration Source”>
    <sources>
      <add name=”File Configuration Source” type=”Microsoft.Practices.EnterpriseLibrary.Common.Configuration.FileConfigurationSource, Microsoft.Practices.EnterpriseLibrary.Common, Version=4.1.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35″
        filePath=”C:\Program Files (x86)\Microsoft BizTalk Server 2009\BizTalkEntLib.config” />
    </sources>
  </enterpriseLibrary.ConfigurationSource>

thats it and you are all set to go!

Bulk insert using WCF SQL Adapter

We had a scenario where we had to do a bulk insert of about 200 msgs into sql, after a lot of branistomring and googling we came up with the following solution

– Create a new table type
CREATE TYPE [dbo].[BulkInsertDBType] AS TABLE(
      [ID] [nvarchar](50) NULL,
      [Creation_Date_Time] [nvarchar](50) NULL
)

– Create a stored proc with  MERGE  

 

Create PROC [dbo].[sp_BulkInsertInsert]
 @List  BulkInsertDBType READONLY
  
AS
      SET NOCOUNT ON
      SET XACT_ABORT ON 
     
      BEGIN TRAN
      MERGE BulkInsertDBTable AS [Target]
USING @List   AS [Source]
ON [Target].[ID] = [Source].[ID]
WHEN NOT MATCHED THEN
    INSERT ( [ID], [Creation_Date_Time] )
    VALUES ( [Source].[ID], [Source].[Creation_Date_Time]);
COMMIT

Conclusion: the new WCF SQL Adapter supports table data type which helps for inserting bulk data!

Configure BizTalk for WCF tracing

Add the following to the biztalk config for tracing all the WCF calls and messages

<system.diagnostics>
    <sources>
      <source name=”System.ServiceModel.MessageLogging”>
        <listeners>
          <add name=”messages”
          type=”System.Diagnostics.XmlWriterTraceListener”
          initializeData=”c:\wcfTrace.e2e” />
        </listeners>
      </source>
    </sources>
  </system.diagnostics>

Simple method to calculate checksum

string str = “123456 string 1344”;

 System.Security.Cryptography.MD5CryptoServiceProvider provider = new System.Security.Cryptography.MD5CryptoServiceProvider();

byte[] byteArray = provider.ComputeHash(System.Text.Encoding.UTF8.GetBytes(str)); double d = BitConverter.ToDouble(byteArray, 0); int i = BitConverter.ToInt32(byteArray, 0);

http://social.msdn.microsoft.com/Forums/en-US/csharpgeneral/thread/5eafd0a2-1fae-437a-aeb1-f3aeca231