Flat File Debatching

Wednesday, November 04, 2009 10:28 PMKWeber

0
Sign In to Vote
Hi

I have to debatch a flat File with the following structure

UN1
.
.
.
UN1
.
.
.
etc.

I would like to do the devbatching with a pipeline.

My Schema has a root node and a record with an element inside

root
record
element (where the data is inside)

I cannot use a fully strutured schema because the the incoming flatfile have a different structure. The only characterisitca they have in common is the header record UN1.

I tired a lot but I did not succeed. How do I have to configure the child order and the child order delimiter type for this flat file?

Thanks for your help
Klaus Weber
- Reply
- Quote

Answers

Wednesday, November 11, 2009 12:10 AMKiryl Kavalenka

1
Sign In to Vote
Prefix means the delimiter preceeds the data, i.e. <delimeter><data><delimeter><data>. This is your case, I don't know why it didn't work - I'll look into it if I have spare time.

When you changed child order to Infix, you "told" parser to extract data from both sides of delimiter, i.e.
<data><delimeter><data><delimeter><data>

As you can see, when the child order is Infix you get 3 <data> elements (messages in your case) off a file with 2 delimiter. The first message will have 0 bytes because the input file starts with the delimiter.
Kiryl Kavalenka My Blog
- Marked As Answer byAndrew_ZhuMSFT, ModeratorThursday, November 12, 2009 2:41 AM
- Reply
- Quote

All Replies

Thursday, November 05, 2009 1:03 PMCarlos T.

0
Sign In to Vote
Try this. Map it to a database all the records you have. Then create another map that will select all the records into another map. This will enable you to break the file appart into single, two, or however many messages you want because you will select the records from a stored procedure. It will do what you need. Good luck.
Please Indicate "Mark as Answer" if a Post has Answered the Question
- Reply
- Quote
Friday, November 06, 2009 2:50 PMKWeber

0
Sign In to Vote
Hi

I tried some other things and I nearly have the solution:

I modified my schema to

<Schema>
       Root
          Element

The configuration for the Child order and delimiters is:

<schema> => Child Delimiter = CR/LF UN1, Default Child Order = Conditional Default, Alllow Message Breakup at Infix Root = Yes
    Root     => Child Delimiter = Default Child Delimiter, Child Order= Infix

The result is that the pipeline debtaches the interchange into the correct number of messages but deletes the UN1 in every message EXCEPT the first one.

Thanks for your thoughts
Klaus Weber
- Reply
- Quote

Saturday, November 07, 2009 10:28 AM

Kiryl Kavalenka

Try this:

<Schema>
Root (Child Order= Postfix, Delimiter = 0xD 0xA, Delimiter Type = Hexadecimal)
Element (Tag Identifier= UN1)

Field

<?xml version="1.0" encoding="utf-16"?>
<xs:schema xmlns:b="http://schemas.microsoft.com/BizTalk/2003" xmlns="http://Test.Schemas.test.FlatFileSchema1" targetNamespace="http://Test.Schemas.test.FlatFileSchema1" xmlns:xs="http://www.w3.org/2001/XMLSchema">
  <xs:annotation>
    <xs:appinfo>
      <b:schemaInfo standard="Flat File" root_reference="Root" default_pad_char=" " pad_char_type="char" count_positions_by_byte="false" parser_optimization="speed" lookahead_depth="3" suppress_empty_nodes="false" generate_empty_nodes="true" allow_early_termination="false" early_terminate_optional_fields="false" allow_message_breakup_of_infix_root="false" compile_parse_tables="false" default_child_order="postfix" />
      <schemaEditorExtension:schemaInfo namespaceAlias="b" extensionClass="Microsoft.BizTalk.FlatFileExtension.FlatFileExtension" standardName="Flat File" xmlns:schemaEditorExtension="http://schemas.microsoft.com/BizTalk/2003/SchemaEditorExtensions" />
    </xs:appinfo>
  </xs:annotation>
  <xs:element name="Root">
    <xs:annotation>
      <xs:appinfo>
        <b:recordInfo structure="delimited" preserve_delimiter_for_empty_data="true" suppress_trailing_delimiters="false" sequence_number="1" child_delimiter_type="hex" child_order="postfix" child_delimiter="0x0D 0x0A" />
      </xs:appinfo>
    </xs:annotation>
    <xs:complexType>
      <xs:sequence>
        <xs:annotation>
          <xs:appinfo>
            <b:groupInfo sequence_number="0" />
          </xs:appinfo>
        </xs:annotation>
        <xs:element name="Record">
          <xs:annotation>
            <xs:appinfo>
              <b:recordInfo sequence_number="1" structure="delimited" preserve_delimiter_for_empty_data="true" suppress_trailing_delimiters="false" tag_name="UN1" />
            </xs:appinfo>
          </xs:annotation>
          <xs:complexType>
            <xs:sequence>
              <xs:annotation>
                <xs:appinfo>
                  <b:groupInfo sequence_number="0" />
                </xs:appinfo>
              </xs:annotation>
              <xs:element name="Field" type="xs:string">
                <xs:annotation>
                  <xs:appinfo>
                    <b:fieldInfo sequence_number="1" justification="left" />
                  </xs:appinfo>
                </xs:annotation>
              </xs:element>
            </xs:sequence>
          </xs:complexType>
        </xs:element>
      </xs:sequence>
    </xs:complexType>
  </xs:element>
</xs:schema>

Monday, November 09, 2009 1:36 PMKWeber

0
Sign In to Vote
Hi Kiryl

Thanks for your input. With your schema I get the error " The remaining stream has unrecognizable data"

Here I have some sample data. Perhaps my problem is easier to understand with this sample

Source:

UN1UNOA212345678990123                                        2222222                                              0000000
UNH1             DESADVD 93AUNEAN004
BGM351                                         500326                             9
DTM13720090703102
DTM17 20090703102
REFON 44444444
REFIV 333333
NADBY 12345678990123
NADSU 12345678990123
NADDP 12345678990123
CPS1
LIN0000014004708355246EN
PIA 1111111                             SA
IMD                   Productinformation short
QTY12 000000000005000PCE
CNT2 000000000000000001
UNT00001500000000000000
UNZ00000516380000000000
UN1UNOA21234567890123                                        2222222                                              0000000
UNH1             DESADVD 93AUNEAN004
BGM351                                         500331                             9
DTM13720090703102
DTM17 20090703102
REFON 44444444
REFIV 333333
NADBY 12345678990123
NADSU 12345678990123
NADDP 12345678990123
CPS1
LIN0000011111111111111EN
PIA 2222222                             SA
IMD                   Productinformation short
QTY12 000000000001000PCE
CNT2 000000000000000001
UNT00001500000000000000

I need two files with the UN1 record at the beginning of each file

Thanks for your help
Klaus Weber
- Reply
- Quote

Monday, November 09, 2009 6:27 PM

Kiryl Kavalenka

Sorry, it is still not clear whether you want "UN1" to be preserved in the output messages or not.
If not, here is the schema:

<Schema>
Root (Child Order= Prefix, Delimiter = UN1, Delimiter Type = Character)
Element

Field

<?xml version="1.0" encoding="utf-16"?>
<xs:schema xmlns:b="http://schemas.microsoft.com/BizTalk/2003" xmlns="http://BizTalk_Server_Project1.FlatFileSchema1" targetNamespace="http://BizTalk_Server_Project1.FlatFileSchema1" xmlns:xs="http://www.w3.org/2001/XMLSchema">
  <xs:annotation>
    <xs:appinfo>
      <b:schemaInfo standard="Flat File" root_reference="Root" default_pad_char=" " pad_char_type="char" count_positions_by_byte="false" parser_optimization="speed" lookahead_depth="3" suppress_empty_nodes="false" generate_empty_nodes="true" allow_early_termination="false" early_terminate_optional_fields="false" allow_message_breakup_of_infix_root="false" compile_parse_tables="false" />
      <schemaEditorExtension:schemaInfo namespaceAlias="b" extensionClass="Microsoft.BizTalk.FlatFileExtension.FlatFileExtension" standardName="Flat File" xmlns:schemaEditorExtension="http://schemas.microsoft.com/BizTalk/2003/SchemaEditorExtensions" />
    </xs:appinfo>
  </xs:annotation>
  <xs:element name="Root">
    <xs:annotation>
      <xs:appinfo>
        <b:recordInfo structure="delimited" preserve_delimiter_for_empty_data="true" suppress_trailing_delimiters="false" sequence_number="1" child_delimiter_type="char" child_delimiter="UN1" child_order="prefix" />
      </xs:appinfo>
    </xs:annotation>
    <xs:complexType>
      <xs:sequence>
        <xs:annotation>
          <xs:appinfo>
            <b:groupInfo sequence_number="0" />
          </xs:appinfo>
        </xs:annotation>
        <xs:element name="Record">
          <xs:annotation>
            <xs:appinfo>
              <b:recordInfo sequence_number="1" structure="delimited" preserve_delimiter_for_empty_data="true" suppress_trailing_delimiters="false" />
            </xs:appinfo>
          </xs:annotation>
          <xs:complexType>
            <xs:sequence>
              <xs:annotation>
                <xs:appinfo>
                  <b:groupInfo sequence_number="0" />
                </xs:appinfo>
              </xs:annotation>
              <xs:element name="Field" type="xs:string">
                <xs:annotation>
                  <xs:appinfo>
                    <b:fieldInfo sequence_number="1" justification="left" />
                  </xs:appinfo>
                </xs:annotation>
              </xs:element>
            </xs:sequence>
          </xs:complexType>
        </xs:element>
      </xs:sequence>
    </xs:complexType>
  </xs:element>
</xs:schema>

Kiryl Kavalenka My Blog

Proposed As Answer byAndrew_ZhuMSFT, ModeratorTuesday, November 10, 2009 2:47 AM

Tuesday, November 10, 2009 9:09 AMKWeber

0
Sign In to Vote
Hi kyril

I would like to to preserve the UN1. I tried your schema but it does not split the interchange inside the pipeline using the flatfile dissasembler.
Klaus Weber
- Reply
- Quote
Tuesday, November 10, 2009 10:54 AMcallvishnu

0
Sign In to Vote
Hi,

BizTalk pipeline does not preserve the Header and Trailer information by default. If you need to use only one or two fields you could possible promote them and then you could use them later if required.

Regards
Vishnu
Vishnu
- Reply
- Quote
Tuesday, November 10, 2009 2:41 PMKWeber

0
Sign In to Vote
Hi

After changing the the root child order from Prefix to infix the source file has been split in 3 files. Two of them have an acceptable content (only the UN1 Tag is missing in the header row) and the third one is a 0 byte file. Where does this 0 byte file come from?
Klaus Weber
- Reply
- Quote
Wednesday, November 11, 2009 12:10 AMKiryl Kavalenka

1
Sign In to Vote
Prefix means the delimiter preceeds the data, i.e. <delimeter><data><delimeter><data>. This is your case, I don't know why it didn't work - I'll look into it if I have spare time.

When you changed child order to Infix, you "told" parser to extract data from both sides of delimiter, i.e.
<data><delimeter><data><delimeter><data>

As you can see, when the child order is Infix you get 3 <data> elements (messages in your case) off a file with 2 delimiter. The first message will have 0 bytes because the input file starts with the delimiter.
Kiryl Kavalenka My Blog
- Marked As Answer byAndrew_ZhuMSFT, ModeratorThursday, November 12, 2009 2:41 AM
- Reply
- Quote
Wednesday, November 11, 2009 12:12 AMKiryl Kavalenka

0
Sign In to Vote
Are you sure that MaxOccurs of "Record" element is set to 1(the default value)?
Kiryl Kavalenka My Blog
- Reply
- Quote
Wednesday, November 11, 2009 11:47 AMKWeber

0
Sign In to Vote
Hi Kyril

Yes I definitely set the MaxOccurence to 1. When I test it in Visual Studio I think the debtaching is ok because only the first message is translated to xml (=> validate instance). If I do it via pipeline the result is one file with both messages
Klaus Weber
- Reply
- Quote