none
Regular expressions and the description field

    Question

  • What am I doing wrong.  I'm looking for Event 4022 with a source of "MSExchange Availability" and the EventDescription matching a regular expression.  I would like to use "CrossSite,\surl\s=\shttps://.*(\.atr...\.|\.md\.)domain\.com/EWS/Exchange\.asmx" 

    HOWEVER, I tried to simplify it down to ".*" and I still get an error logged in the operations manager log file with eventid 10303 which says "the microsoft Operations manager expression filter module failed to process a data item and dropped it."    ERROR 0x80004005

    you cannot get any simpler than ".*".  I am generating the event log entry manually (.net code) to test.  What is it with MOM/SCOM and regular expressions !!!!!!

    I test the expression in a tool "expresso" which is excellent for testing.  "Real" regular expressions NEVER seem to work with MOM/SCOM.

    that being said... I tried looking for a literal string "contained" in the description and it works fine but I want to use the regular expression to be more flexible.  I may end up having to use multiple OR'd contains statements but it really defeats having the option for regular expressions.

    Does anyone have a Regular Expression tester that "thinks" the way the MOM and SCOM engines "think"?

    sorry for grumbling... I fought through regExpressions with MOM 2005 expecting SCOM to work with standard regular expressions.  Error 80004005 must be the classic "Syntax Error" of the 80's

    Craig
    Thursday, November 19, 2009 12:09 AM

Answers

  • That form of regular expression is not supported by that module.  Please try not to cast a legal definition around the term "regular expression" as there are many different regular expression syntax that are perfectly fine.  In this case, think xpath expressions.

    There are some good examples of this on the MVP site as well as on authormps.com.

    Thanks for not grumbling.


    Microsoft Corporation
    Thursday, November 19, 2009 4:38 PM

All replies

  • That form of regular expression is not supported by that module.  Please try not to cast a legal definition around the term "regular expression" as there are many different regular expression syntax that are perfectly fine.  In this case, think xpath expressions.

    There are some good examples of this on the MVP site as well as on authormps.com.

    Thanks for not grumbling.


    Microsoft Corporation
    Thursday, November 19, 2009 4:38 PM
  • sorry for the grumble.  Xpath helps get the text you want to compare against and even use as part of the comparison.  However I'm still left in a quandry since an Xpath is not a regular expression and I don't see why ".*" would not work.  ".*" means any character 0 or more repititions.  that should have worked but instead it logged an event (10303) in the event log "failed to process data item". 

    So with a regular expression against the EventDescription, does that mean I needed to specify an Xpath for the Event description and then the regular expression would have worked? 

    Alternatively, if regular expressions are not valid for that module, is there documentation on what comparisons are allowed for modules since "matches regular expression" is listed in the dropdown for comparison?

    Craig
    Thursday, November 19, 2009 7:06 PM
  • Ok, this forum doesn't take long documents.  Here comes a few parts of a document....

    Regular expression support in SCOM 2007

    Many teams that are authoring management packs may need to include regular expression matching in their discoveries and groups, as well as for pattern matching in expression criteria in monitors and rules.

    There are two different types of regular expression support in the SCOM product, and you have to know which element you are working in to choose the correct one.  Specifically, Group membership calculation and expression filters use distinctly different syntaxes for pattern matching.

    Group Calculation matching criteria

    Group calculation uses PERL regular expression syntax.  By default, the matching is case insensitive, but in the XML you can specify that an expression needs to be case sensitive by way of a special attribute dedicated to specifying that the expression content should be evaluated in a case sensitive way.

    Group Calculation is found in your MP whenever you are using the Group Calc module.

    The GroupCalc expression has an operator called MatchesRegularExpression that is used to create dynamic group membership based on pattern matching expressions.  The implementation of this operator passes the expression found in the MP XML to the SQL call name dbo.fn_MatchesRegularExpression.  If this call returns 0, the match is false.  If the expression returns 1, the match is true.

    GroupCalc also supports two special sub-elements that abstract away a couple of common regex style queries.

    GroupCalc sub element

    Regex Equivalent

    ContainsSubstring

    ^*{O}.*$                ({O} is replaced by the substring)

    MatchesWildcard

    MP expression

    Regex Equivalent

    ?

    .

    *

    .*

    #

    [0-9]

    Table 1:  GroupCalc special functions

    Note:  If either of these two special operators are used, the evaluation will always be case sensitive.

    Expression Filter matching criteria

    Expression filters used in management packs use .NET Regex expression syntax.  A summary of the .NET regular expression syntax elements appears below.  Expression filters are present in your management pack whenever you are using the Expression Eval module.

    Construct

    SCOM Regex

    Any Character

    .

    Character in Range

    [ ]

    Character not in range

    [^ ]

    Beginning of Line

    ^

    End of Line

    $

    Or

    |

    Group

    ( )

    0 or 1 matches

    ?

    0 or more matches

    *

    1 or more matches

    +

    Exactly N matches

    {n}

    Atleast N matches

    {n, }

    Atmost N matches

    { , n}

    N to M Matches

    {n, m}

    New line character

    \n

    Tab character

    \t

     Regular expressions via SDK

    The SCOM SDK has a Matches criteria operator for filtering objects. This operator use the same functionality as MatchesCriteria in the GroupCalc case explained above.

    When using the SDK to construct a criteria expression to find objects in the Ops Manager database, the following syntax elements are valid (see below).  This syntax is useful when creating a criteria expression that includes any of the following elements:

    ·        Comparison operators

    ·        Wildcard characters

    ·        DateTime values

    ·        Integer to XML Enumeration comparisons

    Comparison operators

    You can use comparison operators when constructing a criteria expression. The valid operators are described in the following table:

    Operator

    Description

    Example(s)

    =, ==

    Evaluates to true if the left and right operand are equal.

    Name = 'mymachine.mydomain.com'

    !=, <>

    Evaluates to true if the left and right operand are unequal.

    Name != 'mymachine.mydomain.com'

    >

    Evaluates to true if the left operand is greater than the right operand.

    Severity > 0

    <

    Evaluates to true if the left operand is less than the right operand.

    Severity < 2

    >=

    Evaluates to true if the left operand is greater than or equal to the right operand.

    Severity >= 1

    <=

    Evaluates to true if the left operand is less than or equal to the right operand.

    Severity <= 3

    LIKE

    Evaluates to true if the left operand matches the pattern that is defined by the right operand. Use the characters in the wildcard table later in this topic to define the pattern.

    Name 'LIKE SQL%'

    Evaluates to true if the Name value is "SQLEngine."

    Name LIKE '%SQL%'

    Evaluates to true if the Name value is "MySQLEngine."

    MATCHES

    Evaluates to true if the left operand matches the regular expression defined by the right operand.

    Name MATCHES 'SQL*05'

    Evaluates to true if the Name value is "SQL2005."

    IS NULL

    Evaluates to true if the value of the left operand is null.

    ConnectorId IS NULL

    Evaluates to true if the ConnectorId property does not contain a value.

    IS NOT NULL

    Evaluates to true if the value of the left operand is not null.

    ConnectorId IS NOT NULL

    Evaluates to true if the ConnectorId property contains a value.

    IN

    Evaluates to true if the value of the left operand is in the list of values defined by the right operand.

    Note

    The IN operator is valid for use only with properties of type Guid.

    Id IN ('080F192C-52D2-423D-8953-B3EC8C3CD001', '080F192C-53B2-403D-8753-B3EC8C3CD002')

    Evaluates to true if the value of the Id property is one of the two globally unique identifiers provided in the expression.

    AND

    Evaluates to true if the left and right operands are both true.

    Name = 'SQL%' AND Description LIKE 'MyData%'

    OR

    Evaluates to true if either the left or right operand is true.

    Name = 'SQL%' OR Description LIKE 'MyData%'

    NOT

    Evaluates to true if the right operand is not true.

    NOT (Name = 'IIS' OR Name = 'SQL')

    Table 3: SDK comparison operators


    Microsoft Corporation
    Friday, November 20, 2009 2:35 AM
  • Regular expression support in SCOM 2007 - Part 2 of 2

     

    Wildcards

    The following table defines the wildcard characters you can use to construct a pattern when using the LIKE operator:

    Wildcard

    Description

    Example

    %

    A wildcard that matches any number of characters.

    Name LIKE 'SQL%'

    Evaluates to true if the Name value is "SQLEngine."

    Name LIKE '%SQL%'

    Evaluates to true if the Name value is "MySQLEngine."

    _

    A wildcard that matches a single character.

    Name LIKE 'SQL200_'

    Evaluates to true for the following Name values:

    "SQL2000"

    "SQL2005"

    Note

    The expression evaluates to false for "SQL200" because the symbol _ must match exactly one character in the Name value.

    []

    A wildcard that matches any one character that is enclosed in the character set.

    Note

    Brackets are also used when qualifying references to MonitoringObject properties. For more information, see Defining Queries for Monitoring Objects.

    Name LIKE 'SQL200[05]'

    Evaluates to true for the following Name values:

    "SQL2000"

    "SQL2005"

    The expression evaluates to false for

    "SQL2003."

    [^]

    A wildcard that matches any one character that is not enclosed in the character set.

    Name LIKE 'SQL200[^05]'

    Evaluates to true for

    "SQL2003."

    The expression evaluates to false for

    "SQL2000" and

    "SQL2005."

    Table 4:  Wildcard operators used with LIKE operator

    DateTime comparisons

    When you use a DateTime value in a query expression, use the general DateTime format ("G") to convert the DateTime value to a string value. For example,

    C#

    string qStr = "TimeCreated <= '" + myInstant.ToString("G") + "'";

    ManagementPackCriteria mpCriteria = new ManagementPackCriteria(qStr);

    All date values need to be converted to the G format (GMT) so that valid string comparisons can be made.

    Integer value comparison to enumerations

    When you use an integer enumeration value in a query expression, cast the enumeration value to an integer. For example,

    C#

    string qStr = "Severity > " + (int)ManagementPackAlertSeverity.Warning;

    MonitoringAlertCriteria alertCriteria = new MonitoringAlertCriteria(qStr);


    Microsoft Corporation
    Friday, November 20, 2009 2:36 AM
  • No activity for 30 days, will mark this thread as answered now. Feel free to open it again.
    Best regards, Marnix Wolf

    (Thoughts on OpsMgr)
    Friday, December 18, 2009 7:40 AM
    Moderator
  • This is an old thread but still an issue.  This is a possible solution.

    https://support.microsoft.com/en-us/kb/3004791

    This problem may occur if the length of the line for the match is longer than the Expression Filter can handle.

    To resolve this problem, follow these steps:

        Create the following registry subkey:

        HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft Operations Manager\v3\Modules\Global\ExpressionFilter
        Underneath this subkey, create a DWORD value.
        Type the following name for the DWord value:

        MaxExpressionDepth
        Assign a data value that is between 500 and 100000. The default value is 2000.

    Tuesday, May 26, 2015 9:31 PM