BizTalk Flat File Assembler with Untagged Header and Trailer/Footer

I needed to build an interface which converted an inbound message into a tab delimited file containing 1 header line, any number of detail lines and finally 1 footer line - essentially 3 different schemas/formats in one file E.g.

1->0000002->20130101->M
00001->A bike->300.00->Sent->true->true
00002->A ball->30.00->Placed->true->false
00003->A skipping rope->5.00->Sent->true->true
1->3->335.00

I originally tried combining the 3 different schemas into one and had some success. However the footer row was causing issues and I subsequently found out that BizTalk didn’t know when the repeating record finished and the footer row started so it was processing all the rows as the repeating row and failing with something like “Native Parsing Error: Unexpected data found while looking for:’\t’”. One solution was to add a Tag identifier to the footer schema e.g. FOOTER->1->3->335.00 which worked fine except that this was someone else’s schema (a 3rd party) so I couldn’t start adding FOOTER-> to the end of each file otherwise it wouldn’t get processed.

The clue to the solution was provided in this post by colmac73 which involves the execution of pipeline components in an orchestration. This post adds a few more details as to how I achieved it.

1. First of all create a separate schema for each of the 3 formats, e.g. SchemaHeader, SchemaBody, SchemaTrailer. This is described in detail in a Microsoft article which explains how to do a similar thing but for receiving rather than sending messages.
2. Create 3 separate maps to transform the source data to each of the SchemaHeader, SchemaBody, SchemaTrailer you have just created
3. Create a pipeline object called ConvertToFlatfile containing nothing more than a flat file assembler. As colmac73 says in his article, leave the header / trailer  and document schemas to (none) as these will automatically be detected at runtime.
4. Create an orchestration which receives your inbound message however you prefer. I like to use filter based receive objects that look for schema types, promoted properties or in this case BTS.ReceivedPortName = “MyFileReceivePort”
5. Create 3 messages in the orchestration for each of the SchemaHeader, SchemaBody, SchemaTrailer
6. Create a message called CombinedSendPipelineMessage of type System.Xml.XmlDocument. This will be the message you send to the outbound port.
7. Create a variable called SendPipelineInput of type Microsoft.XLANGs.Pipeline.SendPipelineInputMessages. Note that you will need to reference Microsoft.XLANGs.Pipeline.dll which is located in the BizTalk root installation directory, e.g. D:\Program Files (x86)\Microsoft BizTalk Server 2010
8. Execute all 3 of your transforms either in sequence or in parrallel
9. Create a message assignment shape and enter something like this

   CombinedSendPipelineMessage = new System.Xml.XmlDocument();
   
   SendPipelineInput = new Microsoft.XLANGs.Pipeline.SendPipelineInputMessages();
   SendPipelineInput.Add(msgHeader);
   SendPipelineInput.Add(msgBody);
   SendPipelineInput.Add(msgTrailer);
   
   Microsoft.XLANGs.Pipeline.XLANGPipelineManager.ExecuteSendPipeline(typeof(MyCompany.Namespace.Pipelines.ConvertToFlatfile),SendPipelineInput, CombinedSendPipelineMessage);
   

10. Send the message to a send port. I usually like to use direct subscription / filter based bindings but for this I configured the port directly to the orchestration using “Specify later”
11. Configure the send port as file / SFTP depending on your requirements and select PassThroughTransmit as the pipeline (as the pipeline has already been executed from within the orchestration).
12. Now if everything is working correctly, you should be able to output any inbound message into the required header / body / trailer format at your send port.

Your Orchestration might look a bit like this.