locked
Alert with EventId 33333 and Source DataAccessLayer RRS feed

  • Question

  • Hello!

     

    I have developed an Outbound connector.

    In occasions that are not still clear to me, this error exits in the Operations Manager log of my RMS (root mgmt server):

    EventId: 33333

    Source: DataAccessLayer

    Description: Data Access Layer rejected retry on SqlError:

     Request: AlertUpdate -- (AlertId=8b31fef8-103e-4ffd-b0a6-1b0cb670cc43), (BaseManagedEntityId=ac65b07f-82f4-1492-3925-09cf546b8951), (ResolutionState=0), (Owner=), (CustomField1=), (CustomField2=), (CustomField3=), (CustomField4=), (CustomField5=), (CustomField6=), (CustomField7=), (CustomField8=), (CustomField9=), (CustomField10=), (Comments=Message dispatched successfully), (TimeLastModified=6/8/2009 12:12:11 PM), (ModifiedBy=KRMCMESPE01\Administrator), (TicketId=), (ConnectorId=2f5adcde-631d-412b-bf9f-c200288546b1), (ModifyingConnectorId=), (RETURN_VALUE=-1)

    Class: 16

    Number: 777980200

    Message: Request to update alert ignored due to invalid TimeModified

     

    and my Outbound connector cathces this exception:

    ConnectorMonitoringAlert's Update method failed.

    System.Collections.ListDictionaryInternal

    Request to update alert ignored due to invalid TimeModified

    Microsoft.EnterpriseManagement.OperationsManager

       at Microsoft.EnterpriseManagement.Monitoring.MonitoringAlert.UpdateInternal(String comments, Nullable`1 modifyingConnectorId)

       at Microsoft.EnterpriseManagement.Monitoring.MonitoringAlert.Update(String comments)

       at SCOM_CS_WinSvc.ServiceCore.MonitorState(Object state)

    Void UpdateInternal(System.String, System.Nullable`1[System.Guid])

     

    I have already applied this patch:

    http://support.microsoft.com/kb/957135

     

    But still the error happens.

     

    Some hint ?

    Lizzie!

    Monday, June 8, 2009 12:35 PM

Answers

  • Hey there, I've come late to this discussion, and I admit I haven't read much beyond your initial question, but that very same issue has sprung up for me as well. There's a very small window of time when you can grab an alert and when you can .update it. So if I'm grabbing a bunch of alerts to manipulate, I'll always call the $alert.refresh method before I change anything and update it. This validates that the alert is still valid and fetches the newest timestamp for it.

    Let me know if that fixed it, if not I'll go into more depth about this otmorrow.
    Wednesday, June 24, 2009 8:50 AM
  • Hi Lizzie,

    You need to check the following :

    1. Can you connect to your domain controller from your server? Any auth-related errors in EventLog? 
    2. Sometimes a warning 33333 is issued when not enough RAM on server (RMS)
    3. Account permission issues.  Check if you have any 'access denied' messages in SQL Logs?

    This error indicates that the alert you are trying to update has been updated by someone (or the system) after the UI retrieved the alert but prior to you submitting the update.


    http://OpsMgr.ru/
    • Marked as answer by StuartR Thursday, August 27, 2009 7:55 PM
    Tuesday, June 9, 2009 4:00 AM

All replies

  • Hi Lizzie,

    You need to check the following :

    1. Can you connect to your domain controller from your server? Any auth-related errors in EventLog? 
    2. Sometimes a warning 33333 is issued when not enough RAM on server (RMS)
    3. Account permission issues.  Check if you have any 'access denied' messages in SQL Logs?

    This error indicates that the alert you are trying to update has been updated by someone (or the system) after the UI retrieved the alert but prior to you submitting the update.


    http://OpsMgr.ru/
    • Marked as answer by StuartR Thursday, August 27, 2009 7:55 PM
    Tuesday, June 9, 2009 4:00 AM
  • Hi Alexey,
    1) I can. No authorization related error in EventLog. Can you explain me why these errors are important? The SCOM DB is local and SCOM can work even though with DC not reachable. RMS services are configured as Local System.
    2) Theare are 2 GB of RAM. Prerequisite checks are all green. Canyou explain me why RAM could generate this error?
    3) I did not find any Access Denied (Ichecked only in the Windows Events Logs).
    I am wondering if the environment works perfectly with the other features probably I have not any access denied problems.

    Could you explain me in detail "This error indicates that...etc." ?

    Thank you!
    Lizzie!
    Tuesday, June 9, 2009 8:59 AM
  • 1. Authorization errors are important because if you are not authorized you can't read or write any data. Obviously :) You didn't said local or remote SQL you are using. I saw a warnings 33333 in my lab environment when SQL server got more then 5 minutes time difference with RMS and kerberos didn't worked...
    2. Prerequisite checks only minimal requirements. On the root management server, the most critical resource is RAM followed by CPU. Many of the operations performed on the root management server are memory-intensive. just read that: http://cameronfuller.spaces.live.com/blog/cns!A231E4EB0417CB76!1180.entry

    3. You shold check SQL Server logs not WinEvents
    http://OpsMgr.ru/
    Tuesday, June 9, 2009 9:45 AM
  • 1. This case (luckily)  cannot be applieadto my environment. One possibility less! :-)
    2. Do you mean that even if SCOM itself says that it can work in the server equipped in that way actually it cannot ? This could be a huge problem with our customers. Please, can you confirm this ?
    3. In the SQL Server logs I did not find any authorization events nor "access denied" messages starting from one week ago and this problem happened two days ago.

    What could I do now?

    Thank you!
    Lizzie!

    Tuesday, June 9, 2009 4:41 PM
  • 2. Even if SCOM itself says that it can work in the server equipped it means that OpsMgr just _can work_ and does not mean that OpsMgr will work well on this hardware with any number of agents, management packs, network devices etc. Prerequisite checker checks only minimal requirements like Windows Vista installer checks machine for minimum 512 Mb of RAM. But if so - it does not mean that you can run, for example, CorelDraw on 512 Mb with acceptable performance. When you add a new agent - RMS need some RAM for it, when you add a new connector - RMS need some RAM again and so on.
    Do not use a Prerequisite checker as a planning tool. You should use a SC Capacity Planner (http://technet.microsoft.com/en-us/sccp/default.aspx).
    I have a small lab with All-in-one server (SQL 2008 + OpsMgr RMS) on a box with 2Gb of RAM. Only 5 agents is connected and sometimes I see a performance issues :) SQL Server need a lot of RAM :)

    How many free RAM do you see on RMS now?


    http://OpsMgr.ru/
    Wednesday, June 10, 2009 2:49 AM
  • Hi Alexey!

     

    Good (at least I hope) news! I’ve just isolated the problem but I need to attach two files, there is a way to do it ?

    Wednesday, June 10, 2009 8:47 AM
  • You can upload files to SkyDrive and give us a link :)


    http://OpsMgr.ru/
    Wednesday, June 10, 2009 9:02 AM
  • Hi Alexey!

     

    Good (at least I hope) news! I’ve just isolated the problem:

    1 – In the capture I enclosed you have the history of the alert object of the 33333 event.

    http://cid-55c7e1179864dd7f.skydrive.live.com/self.aspx/.Public/Capture.JPG

    2 – In the alert.txt file you have the get-alert output based on the AlertId

    http://cid-55c7e1179864dd7f.skydrive.live.com/self.aspx/.Public/Alert.txt

    3 – In the EventsDescr.txt you have the descriptions of 33333 and my connector exception events

    http://cid-55c7e1179864dd7f.skydrive.live.com/self.aspx/.Public/ErrorsDescr.txt


    Note:

    Actually I closed many alerts but this problem happened only with a subset of alerts.

    Wednesday, June 10, 2009 1:18 PM
  • Is it possible the exception you are getting is actually legitimate? You would get this exception if from the time you read the alert in your connector to the time you called update on it, the alert was changed by some other process. How long do you keep the alerts cached in your connector? For the alerts that failed, have you tried updating them again either via code or via the UI? If that succeeds I would say it's definitley related to the alert you are updating in code in the connector being stale.
    -- Jakub Oleksy Developer Blog: http://blogs.msdn.com/jakuboleksy/ This posting is provided "AS IS" with no warranties, and confers no rights.
    Wednesday, June 10, 2009 7:21 PM
  • Hi,


    I have the same issue!
    I have 6Go Of RAM, only 4 Go is used!
    No permission issues are logged in SQL log


    Thanks for help
    Lourh
    Wednesday, June 17, 2009 8:13 PM
  • Hello all!
    I have made further tests and I isolated more deeply the problem:
    1 - My connector forward the alerts every 30 seconds
    2 - I stop the router between the SCOM RMS equipped with my Connector (server A) and the server (Server B) that receives the alerts from the Connctor
    3 - I generate (for example) 3 alerts in SCOM RMS
    4 - The connector try every 30 seconds to add the alerts in Server B but the link is down
    5 - I start the router after some minute minutes
    6 - The connector add the alerts in Server B because now the link is up
    --> A - in the Operations Manager of RMS I see two error 33333 related to two of the three alerts
    --> B - In the server B I see 5 alerts not only 3. Two of these five alerts have been sent twice because involved in the 33333 Error.

    The portion of the flow of my connector is:

    1 -  Alert written in SCOM RMS
    2 -  SCOM via Subscription passes the alert to my Connector

    Checking for new alerts

    3 -  For Each alert in the Alert Collection
    4 -    Connector try to add the alert in server B and if Server B is reachable
    5 -        Connector put the alert resolution state in 255
    6 -        Connector add the alert in Server B
    7 -        Connector executes an alert.update ("MyText")
    8 -     end if
    9 -  Next
    10 - Acknowledge Alerts Collection

    Loop for Checking for new alerts

    Could this help ?

    Monday, June 22, 2009 9:52 AM
  • Hey there, I've come late to this discussion, and I admit I haven't read much beyond your initial question, but that very same issue has sprung up for me as well. There's a very small window of time when you can grab an alert and when you can .update it. So if I'm grabbing a bunch of alerts to manipulate, I'll always call the $alert.refresh method before I change anything and update it. This validates that the alert is still valid and fetches the newest timestamp for it.

    Let me know if that fixed it, if not I'll go into more depth about this otmorrow.
    Wednesday, June 24, 2009 8:50 AM
  • Hello!

     

    Prorably your idea is the right solution but I need one more suggestion:

    Currently I have a method that periodically check the alerts for the connector:

     

    alerts = connector.GetMonitoringAlerts();

    if (alerts.Count > 0)

    {

    foreach (ConnectorMonitoringAlert alert in alerts)

                                       {

                           ...do...

    }

     

    }

     

    I was thinking to put the alert.Refresh inside the foreach. Whad do you think ?

    Friday, July 10, 2009 10:30 AM
  • http://opsmgradmin.blogspot.com/2011/05/scom-event-id-33333-in-scom-from-source.html
    Saturday, December 31, 2011 8:56 AM
  • thank you
    Friday, March 21, 2014 9:05 PM