Sign in
Microsoft.com
 
Resources for IT Professionals
 
 
 
Resources for IT Professionals > Forums Home > BizTalk Server Forums > BizTalk Server General > Flat File - optional trailer plus optional delimiter at the end of file
Ask a questionAsk a question
Search Forums:
 

AnswerFlat File - optional trailer plus optional delimiter at the end of file

  • Friday, October 30, 2009 2:46 AMKiryl Kavalenka Users MedalsUsers MedalsUsers MedalsUsers MedalsUsers Medals
     
    Sign In to Vote
    0
    Sign In to Vote
    Hi,

    I have a difficulty creating a flat file schema which would accept both optional trailer record and would be tolerant to missing delimiters at the end of file.

    Instance file examples:

    1) Trailer present + CRLF (i.e. \r\n) present at the end:

    D_TAG data1\r\n
    D_TAG data2\r\n
    T_TAG trailer\r\n

    2) Trailer present + CRLF absent at the end:

    D_TAG data1\r\n
    D_TAG data2\r\n
    T_TAG trailer

    3) Trailer absent + CRLF present at the end:

    D_TAG data1\r\n
    D_TAG data2\r\n

    4) Trailer absent + CRLF absent at the end:

    D_TAG data1\r\n
    D_TAG data2


    Depending on "Child Order" settings of the root node I get the following results:

             
                       Case:      1)      2)     3)      4)
    Child Order:
       Postfix                    OK   FAIL   OK     FAIL
       Infix                       OK    OK    FAIL    OK


    The error when Postfix fails is:
    error BEC2004: Unexpected data found while looking for:
    '\r\n'
    The current definition being parsed is Trailer_Record. The stream offset where the error occured is 31. The line number where the error occured is 3. The column where the error occured is 5.

    The error when Infix fails is:
    error BEC2004: Unexpected end of stream while looking for:
    'T_TAG'
    'D_TAG'
    The current definition being parsed is Root. The stream offset where the error occured is 26. The line number where the error occured is 3. The column where the error occured is 0.
     

Answers

  • Friday, October 30, 2009 4:11 AMAbdul Rafay Users MedalsUsers MedalsUsers MedalsUsers MedalsUsers Medals
     Answer
    Sign In to Vote
    1
    Sign In to Vote
    If you have created the schema that expects CRLF at the end of each record and identify or break records with CRLF so you cannot set it optional there is no such setting. Another solution can be that you can have a custom decode pipeline component that can add CRLF at the end of each record.

    You can create a trailer schema and set it in the first disassembler pipeline component. It will be stripped of if it is present. You can set another disassembler pipeline component in the disassemble stage with no trailer schema. If your document will have the trailer the first disassemble component will be executed or the second one if the trailer is absent.
    Abdul Rafay http://abdulrafaysbiztalk.wordpress.com/ Please mark this as answer if it helps
     
  • Friday, November 06, 2009 4:44 PMLeonid GanelineMVP, ModeratorUsers MedalsUsers MedalsUsers MedalsUsers MedalsUsers Medals
     AnswerHas Code
    Sign In to Vote
    1
    Sign In to Vote
    Kiryl,
    Tested a schema below with all cases. I'm still opposed the solution with custom pipeline component.

    Please, let me know is it what you want.
    <?xml version="1.0" encoding="utf-16"?>
<xs:schema xmlns:b="http://schemas.microsoft.com/BizTalk/2003" xmlns="http://FFSchema.FlatFileSchema1" targetNamespace="http://FFSchema.FlatFileSchema1" xmlns:xs="http://www.w3.org/2001/XMLSchema">
  <xs:annotation>
    <xs:appinfo>
      <schemaEditorExtension:schemaInfo namespaceAlias="b" extensionClass="Microsoft.BizTalk.FlatFileExtension.FlatFileExtension" standardName="Flat File" xmlns:schemaEditorExtension="http://schemas.microsoft.com/BizTalk/2003/SchemaEditorExtensions" />
      <b:schemaInfo standard="Flat File" codepage="20127" default_pad_char=" " pad_char_type="char" count_positions_by_byte="false" parser_optimization="speed" lookahead_depth="3" suppress_empty_nodes="false" generate_empty_nodes="true" allow_early_termination="false" early_terminate_optional_fields="false" allow_message_breakup_of_infix_root="false" compile_parse_tables="false" root_reference="Root" />
    </xs:appinfo>
  </xs:annotation>
  <xs:element name="Root">
    <xs:annotation>
      <xs:appinfo>
        <b:recordInfo structure="delimited" child_order="infix" sequence_number="1" preserve_delimiter_for_empty_data="true" suppress_trailing_delimiters="false" />
      </xs:appinfo>
    </xs:annotation>
    <xs:complexType>
      <xs:sequence>
        <xs:annotation>
          <xs:appinfo>
            <groupInfo sequence_number="0" xmlns="http://schemas.microsoft.com/BizTalk/2003" />
          </xs:appinfo>
        </xs:annotation>
        <xs:element maxOccurs="unbounded" name="D">
          <xs:annotation>
            <xs:appinfo>
              <b:recordInfo structure="positional" tag_name="D_TAG " tag_offset="0" sequence_number="1" preserve_delimiter_for_empty_data="true" suppress_trailing_delimiters="false" />
            </xs:appinfo>
          </xs:annotation>
          <xs:complexType>
            <xs:sequence>
              <xs:annotation>
                <xs:appinfo>
                  <groupInfo sequence_number="0" xmlns="http://schemas.microsoft.com/BizTalk/2003" />
                </xs:appinfo>
              </xs:annotation>
              <xs:element name="Tag" type="xs:string">
                <xs:annotation>
                  <xs:appinfo>
                    <b:fieldInfo justification="left" pos_offset="0" pos_length="6" sequence_number="1" />
                  </xs:appinfo>
                </xs:annotation>
              </xs:element>
              <xs:element name="Data" type="xs:string">
                <xs:annotation>
                  <xs:appinfo>
                    <b:fieldInfo justification="left" pos_offset="0" pos_length="5" sequence_number="2" />
                  </xs:appinfo>
                </xs:annotation>
              </xs:element>
              <xs:element minOccurs="0" name="EOL" type="xs:string">
                <xs:annotation>
                  <xs:appinfo>
                    <b:fieldInfo justification="left" pos_offset="0" pos_length="2" sequence_number="3" />
                  </xs:appinfo>
                </xs:annotation>
              </xs:element>
            </xs:sequence>
          </xs:complexType>
        </xs:element>
        <xs:element minOccurs="0" name="T">
          <xs:annotation>
            <xs:appinfo>
              <b:recordInfo structure="positional" tag_name="T_TAG " tag_offset="0" preserve_delimiter_for_empty_data="true" suppress_trailing_delimiters="false" sequence_number="2" />
            </xs:appinfo>
          </xs:annotation>
          <xs:complexType>
            <xs:sequence>
              <xs:annotation>
                <xs:appinfo>
                  <groupInfo sequence_number="0" xmlns="http://schemas.microsoft.com/BizTalk/2003" />
                </xs:appinfo>
              </xs:annotation>
              <xs:element name="Tag" type="xs:string">
                <xs:annotation>
                  <xs:appinfo>
                    <b:fieldInfo justification="left" pos_offset="0" pos_length="6" sequence_number="1" />
                  </xs:appinfo>
                </xs:annotation>
              </xs:element>
              <xs:element name="Data" type="xs:string">
                <xs:annotation>
                  <xs:appinfo>
                    <b:fieldInfo justification="left" pos_offset="0" pos_length="7" sequence_number="2" />
                  </xs:appinfo>
                </xs:annotation>
              </xs:element>
              <xs:element minOccurs="0" name="EOL" type="xs:string">
                <xs:annotation>
                  <xs:appinfo>
                    <b:fieldInfo justification="left" pos_offset="0" pos_length="2" sequence_number="3" />
                  </xs:appinfo>
                </xs:annotation>
              </xs:element>
            </xs:sequence>
          </xs:complexType>
        </xs:element>
      </xs:sequence>
    </xs:complexType>
  </xs:element>
</xs:schema>
    Leonid Ganeline [BizTalk MVP] My BizTalk blog
     

All Replies

  • Friday, October 30, 2009 4:11 AMAbdul Rafay Users MedalsUsers MedalsUsers MedalsUsers MedalsUsers Medals
     Answer
    Sign In to Vote
    1
    Sign In to Vote
    If you have created the schema that expects CRLF at the end of each record and identify or break records with CRLF so you cannot set it optional there is no such setting. Another solution can be that you can have a custom decode pipeline component that can add CRLF at the end of each record.

    You can create a trailer schema and set it in the first disassembler pipeline component. It will be stripped of if it is present. You can set another disassembler pipeline component in the disassemble stage with no trailer schema. If your document will have the trailer the first disassemble component will be executed or the second one if the trailer is absent.
    Abdul Rafay http://abdulrafaysbiztalk.wordpress.com/ Please mark this as answer if it helps
     
  • Friday, October 30, 2009 6:01 AMBiztalkGhost Users MedalsUsers MedalsUsers MedalsUsers MedalsUsers Medals
     
    Sign In to Vote
    0
    Sign In to Vote
    Hi,

    The below schema works for ur reqt

    Imp : 1. Last elemnt min occurs = 0
             2. Child Order Infix
             3. Preserver delimeter data - yes

      <?xml version="1.0" encoding="utf-16" ?>
    - <xs:schema xmlns:b="http://schemas.microsoft.com/BizTalk/2003" xmlns="http://InvokeDotnetComp.TagSchema_" targetNamespace="http://InvokeDotnetComp.TagSchema_" xmlns:xs="http://www.w3.org/2001/XMLSchema">
    - <xs:annotation>
    - <xs:appinfo>
      <schemaEditorExtension:schemaInfo namespaceAlias="b" extensionClass="Microsoft.BizTalk.FlatFileExtension.FlatFileExtension" standardName="Flat File" xmlns:schemaEditorExtension="http://schemas.microsoft.com/BizTalk/2003/SchemaEditorExtensions" />
      <b:schemaInfo standard="Flat File" codepage="65001" default_pad_char="" pad_char_type="char" count_positions_by_byte="false" parser_optimization="speed" lookahead_depth="3" suppress_empty_nodes="false" generate_empty_nodes="true" allow_early_termination="false" early_terminate_optional_fields="false" allow_message_breakup_of_infix_root="false" compile_parse_tables="false" root_reference="Root" />
      </xs:appinfo>
      </xs:annotation>
    - <xs:element name="Root">
    - <xs:annotation>
    - <xs:appinfo>
      <b:recordInfo structure="delimited" child_delimiter_type="hex" child_delimiter="0xD 0xA" child_order="infix" sequence_number="1" preserve_delimiter_for_empty_data="true" suppress_trailing_delimiters="false" />
      </xs:appinfo>
      </xs:annotation>
    - <xs:complexType>
    - <xs:sequence>
    - <xs:annotation>
    - <xs:appinfo>
      <groupInfo sequence_number="0" xmlns="http://schemas.microsoft.com/BizTalk/2003" />
      </xs:appinfo>
      </xs:annotation>
    - <xs:element name="Root_Child1" type="xs:string">
    - <xs:annotation>
    - <xs:appinfo>
      <b:fieldInfo justification="left" sequence_number="1" />
      </xs:appinfo>
      </xs:annotation>
      </xs:element>
    - <xs:element minOccurs="1" name="Root_Child2" type="xs:string">
    - <xs:annotation>
    - <xs:appinfo>
      <b:fieldInfo justification="left" sequence_number="2" />
      </xs:appinfo>
      </xs:annotation>
      </xs:element>
    - <xs:element minOccurs="0" name="Root_Child3" type="xs:string">
    - <xs:annotation>
    - <xs:appinfo>
      <b:fieldInfo justification="left" sequence_number="3" />
      </xs:appinfo>
      </xs:annotation>
      </xs:element>
      </xs:sequence>
      </xs:complexType>
      </xs:element>
      </xs:schema>

    Vijaykishan | http://v2attechhorizon.blogspot.com | Please mark this as answer if it helps
     
  • Friday, October 30, 2009 6:18 AMDevsap Users MedalsUsers MedalsUsers MedalsUsers MedalsUsers Medals
     
    Sign In to Vote
    0
    Sign In to Vote

    I just did a small prototype for your scenario and it looks like it is working. Basically i am using Sequence group and Choice group(for trailer) and created separate optional record to read the \r\n at the end.

    I tested, it is working for all the scenarios. But not sure your file has tag identifiers.

    Let me know, if it helps.

    Below is the schema i created for my prototype. you can take and paste in a file and save as xsd.




    <?xml version="1.0" encoding="utf-16"?>
    <xs:schema xmlns:b="http://schemas.microsoft.com/BizTalk/2003" xmlns="http://BizTalk_Server_Project1.FlatFileSchema1" targetNamespace="http://BizTalk_Server_Project1.FlatFileSchema1" xmlns:xs="http://www.w3.org/2001/XMLSchema">
      <xs:annotation>
        <xs:appinfo>
          <b:schemaInfo standard="Flat File" root_reference="Root" default_pad_char=" " pad_char_type="char" count_positions_by_byte="false" parser_optimization="speed" lookahead_depth="3" suppress_empty_nodes="false" generate_empty_nodes="true" allow_early_termination="false" early_terminate_optional_fields="false" allow_message_breakup_of_infix_root="false" compile_parse_tables="false" />
          <schemaEditorExtension:schemaInfo namespaceAlias="b" extensionClass="Microsoft.BizTalk.FlatFileExtension.FlatFileExtension" standardName="Flat File" xmlns:schemaEditorExtension="http://schemas.microsoft.com/BizTalk/2003/SchemaEditorExtensions" />
        </xs:appinfo>
      </xs:annotation>
      <xs:element name="Root">
        <xs:annotation>
          <xs:appinfo>
            <b:recordInfo structure="delimited" preserve_delimiter_for_empty_data="true" suppress_trailing_delimiters="false" sequence_number="1" child_delimiter_type="hex" child_delimiter="0x0D 0x0A" child_order="infix" />
          </xs:appinfo>
        </xs:annotation>
        <xs:complexType>
          <xs:choice minOccurs="0">
            <xs:annotation>
              <xs:appinfo>
                <b:groupInfo sequence_number="0" />
              </xs:appinfo>
            </xs:annotation>
            <xs:sequence minOccurs="0">
              <xs:annotation>
                <xs:appinfo>
                  <b:groupInfo sequence_number="1" />
                </xs:appinfo>
              </xs:annotation>
              <xs:element maxOccurs="unbounded" name="CLM">
                <xs:annotation>
                  <xs:appinfo>
                    <b:recordInfo structure="delimited" preserve_delimiter_for_empty_data="true" suppress_trailing_delimiters="false" child_delimiter_type="char" child_order="infix" child_delimiter="," tag_name="CLM" sequence_number="1" />
                  </xs:appinfo>
                </xs:annotation>
                <xs:complexType>
                  <xs:attribute name="Field1" type="xs:string">
                    <xs:annotation>
                      <xs:appinfo>
                        <b:fieldInfo sequence_number="1" justification="left" />
                      </xs:appinfo>
                    </xs:annotation>
                  </xs:attribute>
                  <xs:attribute name="Field2" type="xs:string">
                    <xs:annotation>
                      <xs:appinfo>
                        <b:fieldInfo sequence_number="2" justification="left" />
                      </xs:appinfo>
                    </xs:annotation>
                  </xs:attribute>
                </xs:complexType>
              </xs:element>
              <xs:choice minOccurs="0">
                <xs:annotation>
                  <xs:appinfo>
                    <b:groupInfo sequence_number="2" />
                  </xs:appinfo>
                </xs:annotation>
                <xs:element minOccurs="0" name="TRL">
                  <xs:annotation>
                    <xs:appinfo>
                      <b:recordInfo structure="delimited" preserve_delimiter_for_empty_data="true" suppress_trailing_delimiters="false" child_delimiter_type="char" child_delimiter="," child_order="infix" tag_name="TRL" sequence_number="1" />
                    </xs:appinfo>
                  </xs:annotation>
                  <xs:complexType>
                    <xs:attribute name="Field1" type="xs:string">
                      <xs:annotation>
                        <xs:appinfo>
                          <b:fieldInfo sequence_number="1" justification="left" />
                        </xs:appinfo>
                      </xs:annotation>
                    </xs:attribute>
                    <xs:attribute name="Field2" type="xs:string">
                      <xs:annotation>
                        <xs:appinfo>
                          <b:fieldInfo sequence_number="2" justification="left" />
                        </xs:appinfo>
                      </xs:annotation>
                    </xs:attribute>
                  </xs:complexType>
                </xs:element>
              </xs:choice>
              <xs:element minOccurs="0" name="Record">
                <xs:annotation>
                  <xs:appinfo>
                    <b:recordInfo structure="delimited" preserve_delimiter_for_empty_data="true" suppress_trailing_delimiters="false" sequence_number="3" />
                  </xs:appinfo>
                </xs:annotation>
                <xs:complexType>
                  <xs:sequence>
                    <xs:annotation>
                      <xs:appinfo>
                        <b:groupInfo sequence_number="0" />
                      </xs:appinfo>
                    </xs:annotation>
                    <xs:element minOccurs="0" name="Field" type="xs:string">
                      <xs:annotation>
                        <xs:appinfo>
                          <b:fieldInfo sequence_number="1" justification="left" />
                        </xs:appinfo>
                      </xs:annotation>
                    </xs:element>
                  </xs:sequence>
                </xs:complexType>
              </xs:element>
            </xs:sequence>
          </xs:choice>
        </xs:complexType>
      </xs:element>
    </xs:schema>
     
  • Sunday, November 01, 2009 10:55 PMKiryl Kavalenka Users MedalsUsers MedalsUsers MedalsUsers MedalsUsers Medals
     
    Sign In to Vote
    0
    Sign In to Vote
    Sorry, Devsap, your schema simply does not work for the case 3).
    It says:
    error BEC2004: Unrecognized data in remaining stream
     
  • Sunday, November 01, 2009 11:00 PMKiryl Kavalenka Users MedalsUsers MedalsUsers MedalsUsers MedalsUsers Medals
     
    Sign In to Vote
    0
    Sign In to Vote
    Vijaykishan,

    Your schema a bit to simple (i.e. dumping the whole file into single record and dumping each line into single field). I'm afraid making it a bit more complex will prevent it from accepting all 4 test file instances.

    Here is my schema which is a bit more complex but does not work in case3:

    <?xml version="1.0" encoding="utf-16"?>
    <xs:schema xmlns:b="http://schemas.microsoft.com/BizTalk/2003" xmlns="http://MFL.Schemas.Schemas.Test" targetNamespace="http://MFL.Schemas.Schemas.Test" xmlns:xs="http://www.w3.org/2001/XMLSchema">
      <xs:annotation>
        <xs:appinfo>
          <schemaEditorExtension:schemaInfo namespaceAlias="b" extensionClass="Microsoft.BizTalk.FlatFileExtension.FlatFileExtension" standardName="Flat File" xmlns:schemaEditorExtension="http://schemas.microsoft.com/BizTalk/2003/SchemaEditorExtensions" />
          <b:schemaInfo standard="Flat File" codepage="65001" default_pad_char=" " pad_char_type="char" count_positions_by_byte="false" parser_optimization="complexity" lookahead_depth="0" suppress_empty_nodes="false" generate_empty_nodes="true" allow_early_termination="true" early_terminate_optional_fields="false" allow_message_breakup_of_infix_root="false" compile_parse_tables="false" root_reference="Root" />
        </xs:appinfo>
      </xs:annotation>
      <xs:element name="Root">
        <xs:annotation>
          <xs:appinfo>
            <b:recordInfo structure="delimited" child_delimiter_type="hex" child_delimiter="0xD 0xA" child_order="infix" sequence_number="1" preserve_delimiter_for_empty_data="true" suppress_trailing_delimiters="false" />
          </xs:appinfo>
        </xs:annotation>
        <xs:complexType>
          <xs:sequence>
            <xs:annotation>
              <xs:appinfo>
                <groupInfo sequence_number="0" xmlns="http://schemas.microsoft.com/BizTalk/2003" />
              </xs:appinfo>
            </xs:annotation>
            <xs:element maxOccurs="unbounded" name="Data_Record">
              <xs:annotation>
                <xs:appinfo>
                  <b:recordInfo structure="positional" tag_name="D_TAG" tag_offset="0" sequence_number="1" preserve_delimiter_for_empty_data="true" suppress_trailing_delimiters="false" />
                </xs:appinfo>
              </xs:annotation>
              <xs:complexType>
                <xs:sequence>
                  <xs:annotation>
                    <xs:appinfo>
                      <groupInfo sequence_number="0" xmlns="http://schemas.microsoft.com/BizTalk/2003" />
                    </xs:appinfo>
                  </xs:annotation>
                  <xs:element name="Tag" type="xs:string">
                    <xs:annotation>
                      <xs:appinfo>
                        <b:fieldInfo justification="left" pos_offset="0" pos_length="5" sequence_number="1" />
                      </xs:appinfo>
                    </xs:annotation>
                  </xs:element>
                  <xs:element name="Field" type="xs:string">
                    <xs:annotation>
                      <xs:appinfo>
                        <fieldInfo sequence_number="2" justification="left" pos_offset="0" pos_length="6" xmlns="http://schemas.microsoft.com/BizTalk/2003" />
                      </xs:appinfo>
                    </xs:annotation>
                  </xs:element>
                </xs:sequence>
              </xs:complexType>
            </xs:element>
            <xs:element minOccurs="0" name="Trailer_Record">
              <xs:annotation>
                <xs:appinfo>
                  <b:recordInfo structure="positional" tag_name="T_TAG" tag_offset="0" sequence_number="2" preserve_delimiter_for_empty_data="true" suppress_trailing_delimiters="false" />
                </xs:appinfo>
              </xs:annotation>
              <xs:complexType>
                <xs:sequence>
                  <xs:annotation>
                    <xs:appinfo>
                      <groupInfo sequence_number="0" xmlns="http://schemas.microsoft.com/BizTalk/2003" />
                    </xs:appinfo>
                  </xs:annotation>
                  <xs:element name="Tag" type="xs:string">
                    <xs:annotation>
                      <xs:appinfo>
                        <b:fieldInfo justification="left" pos_offset="0" pos_length="5" sequence_number="1" />
                      </xs:appinfo>
                    </xs:annotation>
                  </xs:element>
                  <xs:element name="Field" type="xs:string">
                    <xs:annotation>
                      <xs:appinfo>
                        <b:fieldInfo justification="left" pos_offset="0" pos_length="8" sequence_number="2" />
                      </xs:appinfo>
                    </xs:annotation>
                  </xs:element>
                </xs:sequence>
              </xs:complexType>
            </xs:element>
          </xs:sequence>
        </xs:complexType>
      </xs:element>
    </xs:schema>
     
  • Monday, November 02, 2009 12:47 AMKiryl Kavalenka Users MedalsUsers MedalsUsers MedalsUsers MedalsUsers Medals
     
    Sign In to Vote
    0
    Sign In to Vote
    Thank you, Abdul.

    1) Having a custom decode pipeline component that adds CRLF at the end of file will fix the problem, but it would be nice not to tamper the original data if possible.

    2) Creating multiple disassembler pipeline components... - The thing is that I already have a custom flat file disassembler component which determines the type of the message and therefore the schema that should be used for parsing to XML.

     
  • Monday, November 02, 2009 9:51 AMAbdul Rafay Users MedalsUsers MedalsUsers MedalsUsers MedalsUsers Medals
     
    Sign In to Vote
    1
    Sign In to Vote
    Thank you, Abdul.

    1) Having a custom decode pipeline component that adds CRLF at the end of file will fix the problem, but it would be nice not to tamper the original data if possible.

    2) Creating multiple disassembler pipeline components... - The thing is that I already have a custom flat file disassembler component which determines the type of the message and therefore the schema that should be used for parsing to XML.


    Kiryl,

    Having a custom decode pipeline component will normalize the data and not modify the data. This is an enterprise integration pattern solution for normalizing data to the required format. Of course you cannot handle these cases yourself unless you have a decoder component that normalizes the data and ensures it has no errors for processing.
    Abdul Rafay http://abdulrafaysbiztalk.wordpress.com/ Please mark this as answer if it helps
     
  • Monday, November 02, 2009 5:59 PMLeonid GanelineMVP, ModeratorUsers MedalsUsers MedalsUsers MedalsUsers MedalsUsers Medals
     Has Code
    Sign In to Vote
    0
    Sign In to Vote
    Hi Kiryl,

    The idea is simple.
    Records and Trailer could be easily recognized by tags (forget of CR LF). Then I added the CR LF as a "noise", I mean the "Pad characters" property on the schema node.
    The result is below (I used the FF Wizard then changed the schema manualy).
    Let me know if it is what you want. The schema works for all your cases.
    <?xml version="1.0" encoding="utf-16"?>
<xs:schema xmlns:b="http://schemas.microsoft.com/BizTalk/2003" xmlns="http://FlatFileSchema2" targetNamespace="http://FlatFileSchema2" xmlns:xs="http://www.w3.org/2001/XMLSchema">
  <xs:annotation>
    <xs:appinfo>
      <schemaEditorExtension:schemaInfo namespaceAlias="b" extensionClass="Microsoft.BizTalk.FlatFileExtension.FlatFileExtension" standardName="Flat File" xmlns:schemaEditorExtension="http://schemas.microsoft.com/BizTalk/2003/SchemaEditorExtensions" />
      <b:schemaInfo standard="Flat File" codepage="20127" default_pad_char="0x0A 0x0D" pad_char_type="hex" count_positions_by_byte="false" parser_optimization="speed" lookahead_depth="3" suppress_empty_nodes="false" generate_empty_nodes="true" allow_early_termination="false" early_terminate_optional_fields="false" allow_message_breakup_of_infix_root="false" compile_parse_tables="false" root_reference="Root" />
    </xs:appinfo>
  </xs:annotation>
  <xs:element name="Root">
    <xs:annotation>
      <xs:appinfo>
        <b:recordInfo structure="delimited" child_order="infix" sequence_number="1" preserve_delimiter_for_empty_data="true" suppress_trailing_delimiters="false" />
      </xs:appinfo>
    </xs:annotation>
    <xs:complexType>
      <xs:sequence>
        <xs:annotation>
          <xs:appinfo>
            <groupInfo sequence_number="0" xmlns="http://schemas.microsoft.com/BizTalk/2003" />
          </xs:appinfo>
        </xs:annotation>
        <xs:element maxOccurs="unbounded" name="MyRecord">
          <xs:annotation>
            <xs:appinfo>
              <b:recordInfo tag_name="D_TAG" structure="delimited" child_order="infix" sequence_number="1" preserve_delimiter_for_empty_data="true" suppress_trailing_delimiters="false" />
            </xs:appinfo>
          </xs:annotation>
          <xs:complexType>
            <xs:sequence>
              <xs:annotation>
                <xs:appinfo>
                  <groupInfo sequence_number="0" xmlns="http://schemas.microsoft.com/BizTalk/2003" />
                </xs:appinfo>
              </xs:annotation>
              <xs:element name="MyRecordData" type="xs:string">
                <xs:annotation>
                  <xs:appinfo>
                    <b:fieldInfo justification="left" sequence_number="1" />
                  </xs:appinfo>
                </xs:annotation>
              </xs:element>
            </xs:sequence>
          </xs:complexType>
        </xs:element>
        <xs:element name="Trailer">
          <xs:annotation>
            <xs:appinfo>
              <recordInfo sequence_number="2" structure="delimited" preserve_delimiter_for_empty_data="true" suppress_trailing_delimiters="false" child_order="infix" tag_name="T_TAG" xmlns="http://schemas.microsoft.com/BizTalk/2003" />
            </xs:appinfo>
          </xs:annotation>
          <xs:complexType>
            <xs:sequence>
              <xs:annotation>
                <xs:appinfo>
                  <groupInfo sequence_number="0" xmlns="http://schemas.microsoft.com/BizTalk/2003" />
                </xs:appinfo>
              </xs:annotation>
              <xs:element name="TrailerData" type="xs:string">
                <xs:annotation>
                  <xs:appinfo>
                    <fieldInfo sequence_number="1" justification="left" xmlns="http://schemas.microsoft.com/BizTalk/2003" />
                  </xs:appinfo>
                </xs:annotation>
              </xs:element>
            </xs:sequence>
          </xs:complexType>
        </xs:element>
      </xs:sequence>
    </xs:complexType>
  </xs:element>
</xs:schema>
    Leonid Ganeline [BizTalk MVP] My BizTalk blog
     
  • Monday, November 02, 2009 10:32 PMKiryl Kavalenka Users MedalsUsers MedalsUsers MedalsUsers MedalsUsers Medals
     
    Sign In to Vote
    0
    Sign In to Vote
    Hi Leonid,
    thanks for the reply.

    Your schema does not validate messages without trailers (cases 3 and 4).

    Also, the structure of Data and Trailer records is "positional", not "delimeted".

    The error is:
    warning BEC2004: Record "Root" - The Child Delimiter Type property is not set for a delimited record.
    +
    error BEC2004: Unexpected data found while looking for:
    'T_TAG'
    'D_TAG'
    The current definition being parsed is MyRecord. The stream offset where the error occured is 18. The line number where the error occured is 2. The column where the error occured is 5.

    (Test data:
    D_TAG data1\r\n
    D_TAG data2\r\n)
     
  • Tuesday, November 03, 2009 4:19 PMLeonid GanelineMVP, ModeratorUsers MedalsUsers MedalsUsers MedalsUsers MedalsUsers Medals
     
    Sign In to Vote
    1
    Sign In to Vote
    Oops...

    Just make the trailer as an optional. It should work.
    You can ignore the warning "warning BEC2004: Record "Root" - The Child Delimiter Type property is not set for a delimited record." because the childs (Record and Trailer) are all explicitly defined as delimited.
    Leonid Ganeline [BizTalk MVP] My BizTalk blog
     
  • Tuesday, November 03, 2009 9:54 PMKiryl Kavalenka Users MedalsUsers MedalsUsers MedalsUsers MedalsUsers Medals
     
    Sign In to Vote
    0
    Sign In to Vote
    Ok, thanks to everybody who replied for your help.

    I ended up using workaround suggested by Abdul. I have created custom decode component which adds CRLF characters to the end of messages. Therefore I have eliminated cases 2) and 4) (i.e. postfix child order now works in all cases).

    I have used FixMsg solution from Microsoft's samples (C:\Program Files\Microsoft BizTalk Server 2006\SDK\Samples\Pipelines\CustomComponent\FixMsg)
    as a template for the decode component.
     
  • Wednesday, November 04, 2009 4:42 PMLeonid GanelineMVP, ModeratorUsers MedalsUsers MedalsUsers MedalsUsers MedalsUsers Medals
     
    Sign In to Vote
    0
    Sign In to Vote
    To use the custom pipeline component for such simple task is overdose, Kiryl.
    Simple schema (as mentioned before) works just fine. I tested it on all your cases. (Do not forget to change the Trailer record's "Min Occurs" to 0.)
    Leonid Ganeline [BizTalk MVP] My BizTalk blog
     
  • Friday, November 06, 2009 1:50 AMKiryl Kavalenka Users MedalsUsers MedalsUsers MedalsUsers MedalsUsers Medals
     
    Sign In to Vote
    0
    Sign In to Vote
    In my case the structure of Data and Trailer records is "positional", not "delimeted". This makes a difference.
     
  • Friday, November 06, 2009 4:44 PMLeonid GanelineMVP, ModeratorUsers MedalsUsers MedalsUsers MedalsUsers MedalsUsers Medals
     AnswerHas Code
    Sign In to Vote
    1
    Sign In to Vote
    Kiryl,
    Tested a schema below with all cases. I'm still opposed the solution with custom pipeline component.

    Please, let me know is it what you want.
    <?xml version="1.0" encoding="utf-16"?>
<xs:schema xmlns:b="http://schemas.microsoft.com/BizTalk/2003" xmlns="http://FFSchema.FlatFileSchema1" targetNamespace="http://FFSchema.FlatFileSchema1" xmlns:xs="http://www.w3.org/2001/XMLSchema">
  <xs:annotation>
    <xs:appinfo>
      <schemaEditorExtension:schemaInfo namespaceAlias="b" extensionClass="Microsoft.BizTalk.FlatFileExtension.FlatFileExtension" standardName="Flat File" xmlns:schemaEditorExtension="http://schemas.microsoft.com/BizTalk/2003/SchemaEditorExtensions" />
      <b:schemaInfo standard="Flat File" codepage="20127" default_pad_char=" " pad_char_type="char" count_positions_by_byte="false" parser_optimization="speed" lookahead_depth="3" suppress_empty_nodes="false" generate_empty_nodes="true" allow_early_termination="false" early_terminate_optional_fields="false" allow_message_breakup_of_infix_root="false" compile_parse_tables="false" root_reference="Root" />
    </xs:appinfo>
  </xs:annotation>
  <xs:element name="Root">
    <xs:annotation>
      <xs:appinfo>
        <b:recordInfo structure="delimited" child_order="infix" sequence_number="1" preserve_delimiter_for_empty_data="true" suppress_trailing_delimiters="false" />
      </xs:appinfo>
    </xs:annotation>
    <xs:complexType>
      <xs:sequence>
        <xs:annotation>
          <xs:appinfo>
            <groupInfo sequence_number="0" xmlns="http://schemas.microsoft.com/BizTalk/2003" />
          </xs:appinfo>
        </xs:annotation>
        <xs:element maxOccurs="unbounded" name="D">
          <xs:annotation>
            <xs:appinfo>
              <b:recordInfo structure="positional" tag_name="D_TAG " tag_offset="0" sequence_number="1" preserve_delimiter_for_empty_data="true" suppress_trailing_delimiters="false" />
            </xs:appinfo>
          </xs:annotation>
          <xs:complexType>
            <xs:sequence>
              <xs:annotation>
                <xs:appinfo>
                  <groupInfo sequence_number="0" xmlns="http://schemas.microsoft.com/BizTalk/2003" />
                </xs:appinfo>
              </xs:annotation>
              <xs:element name="Tag" type="xs:string">
                <xs:annotation>
                  <xs:appinfo>
                    <b:fieldInfo justification="left" pos_offset="0" pos_length="6" sequence_number="1" />
                  </xs:appinfo>
                </xs:annotation>
              </xs:element>
              <xs:element name="Data" type="xs:string">
                <xs:annotation>
                  <xs:appinfo>
                    <b:fieldInfo justification="left" pos_offset="0" pos_length="5" sequence_number="2" />
                  </xs:appinfo>
                </xs:annotation>
              </xs:element>
              <xs:element minOccurs="0" name="EOL" type="xs:string">
                <xs:annotation>
                  <xs:appinfo>
                    <b:fieldInfo justification="left" pos_offset="0" pos_length="2" sequence_number="3" />
                  </xs:appinfo>
                </xs:annotation>
              </xs:element>
            </xs:sequence>
          </xs:complexType>
        </xs:element>
        <xs:element minOccurs="0" name="T">
          <xs:annotation>
            <xs:appinfo>
              <b:recordInfo structure="positional" tag_name="T_TAG " tag_offset="0" preserve_delimiter_for_empty_data="true" suppress_trailing_delimiters="false" sequence_number="2" />
            </xs:appinfo>
          </xs:annotation>
          <xs:complexType>
            <xs:sequence>
              <xs:annotation>
                <xs:appinfo>
                  <groupInfo sequence_number="0" xmlns="http://schemas.microsoft.com/BizTalk/2003" />
                </xs:appinfo>
              </xs:annotation>
              <xs:element name="Tag" type="xs:string">
                <xs:annotation>
                  <xs:appinfo>
                    <b:fieldInfo justification="left" pos_offset="0" pos_length="6" sequence_number="1" />
                  </xs:appinfo>
                </xs:annotation>
              </xs:element>
              <xs:element name="Data" type="xs:string">
                <xs:annotation>
                  <xs:appinfo>
                    <b:fieldInfo justification="left" pos_offset="0" pos_length="7" sequence_number="2" />
                  </xs:appinfo>
                </xs:annotation>
              </xs:element>
              <xs:element minOccurs="0" name="EOL" type="xs:string">
                <xs:annotation>
                  <xs:appinfo>
                    <b:fieldInfo justification="left" pos_offset="0" pos_length="2" sequence_number="3" />
                  </xs:appinfo>
                </xs:annotation>
              </xs:element>
            </xs:sequence>
          </xs:complexType>
        </xs:element>
      </xs:sequence>
    </xs:complexType>
  </xs:element>
</xs:schema>
    Leonid Ganeline [BizTalk MVP] My BizTalk blog
     
  • Saturday, November 07, 2009 7:50 AMKiryl Kavalenka Users MedalsUsers MedalsUsers MedalsUsers MedalsUsers Medals
     
    Sign In to Vote
    0
    Sign In to Vote
    Hmm... Interesting approach. One of those "think outside the box" cases, isn't it? Treating CRLFs not as delimeters but as data while making the most out of tags.

    You're legend, Leonid. Thank you.