none
Unknown warning: 'Max Concurrent API Reached alert'

    Question

  • I started getting these alerts fairly frequently for our Exchange CAS (running Windows 2008 Standard) and I have no idea what is triggering it. It's presumably related to Netlogon events.

    The only info I found was here: http://www.systemcentercentral.com/updated-windows-operating-system-management-pack-released-scom-sysctr/

    The registry key referenced does not exist on the server throwing the alert. I could not correlate the warnings with anything in the Event Logs, either.

    Any information would be appreciated. Thank you.

    Friday, May 24, 2013 2:03 PM

Answers

  • Hi Justin,

    >Secure Channel Error

    >ErrorDescription Could not Read Category Index: 1848.

    The monitor should be disabled for this server(s) due to a bug.


    http://OpsMgr.ru/

    • Marked as answer by j-gray Monday, June 03, 2013 8:16 PM
    Monday, June 03, 2013 5:51 PM
    Moderator

All replies

  • It's calculated from a few perf counters. Please read the knowledge base article for this monitor (in the alert details).

    http://OpsMgr.ru/

    Saturday, May 25, 2013 7:28 AM
    Moderator
  • Thanks for the reply. I've been running perfmon on the system for several days and following the formula referenced in the KB, the computed max concurrent api setting comes in at less than 1.  I've gradually bumped the setting up to 8 in the registry (stopping/starting netlogon after each change) and continue to get the alerts repeatedly.

    The perfmon counters for semaphore holders, timeouts, and waiters all average and max at zero. The max average hold time is .006 over multiple hours when I'm receiving these alerts. I don't find any API-related errors on the netlogon log, either.

    Any other insights?

     
    Thursday, May 30, 2013 6:03 PM
  • What do you see in the alert context?

    http://OpsMgr.ru/

    Friday, May 31, 2013 3:56 AM
    Moderator
  • What do you see in the alert context?

    http://OpsMgr.ru/

    I'm not sure what you are asking. The specific alert is, "Alert description: Max Concurrent API reached in Server xxxx"

    The context of the alert indicates the warning is due to a low default value for MaxConcurrentAPI, exacerbated by forests with multiple domains. The referenced KB includes a formula for determining the MaxConcurrentAPI setting, however, my results indicated a value of less than 1 should suffice. I've raised the value to 8 and still get the alerts. Documentation indicates that raising the value to 10 or higher is not advised as it could create other performance issues.

    Per other documentation I've found on the issue, I've ruled out network latency and netlogon logs are not reflecting any issues.

    At this point, it would seem that the conditions that trigger the alert are not being met, yet I'm hesitant to disregard the alerts.

    Friday, May 31, 2013 2:39 PM
  • Hello Alexey,

    I have 2 types of alert context for the "Max Concurrent API reached" alert:

    -------------------

    Type 1:

    Date and Time: 27/05/2013 17:36:12
    Property Name Property Value
    Detection Time 05/27/2013 17:36:12
    Secure Channel Error
    Semaphore Timeouts 0
    Semaphore Waiters 0
    Semaphore Acquires 0
    Semaphore Holders 0
    Average Semaphore Hold Time 05/27/2013 17:36:12
    State 0
    ErrorNumber 
    ErrorDescription Category does not exist.

    --------------------
    Type 2:

    Date and Time: 31/05/2013 16:15:50
    Property Name Property Value
    Detection Time 05/31/2013 16:15:50
    Secure Channel _Total
    Semaphore Timeouts 0
    Semaphore Waiters 0
    Semaphore Acquires 401
    Semaphore Holders 0
    Average Semaphore Hold Time 0
    State 1
    ErrorNumber 0
    ErrorDescription

    Friday, May 31, 2013 4:10 PM
  • Hi,

    >ErrorDescription Category does not exist.

    This monitor has the bug and this server (source of this alert) is affected. Just ignore this alert (or disable the monitor for this server) until the bug is fixed (read: until the next MP release).

    >Semaphore Acquires 401

    According to the formula from the monitor: ((($SemWaiters -gt 0) -and (-not($SemWaiters -gt 4GB))) -or (($SemHolders -gt 0) -and (-not($SemHolders -gt 4GB))) -or (($SemTimeouts -gt 0) -and (-not($SemTimeouts -gt 4GB))))

    the result is True, so alert is generated.


    http://OpsMgr.ru/

    • Proposed as answer by HermanoF Monday, June 03, 2013 4:34 PM
    Friday, May 31, 2013 4:57 PM
    Moderator
  • Thank you!

    Regards.

    Monday, June 03, 2013 8:58 AM
  • Hello again Alexey, today i received alerts with different context. I suppose the first one is due to the bug,  but in the second all values are 0 except the state parameter. Do you know what that means?
    ----------------
    Date and Time: 03/06/2013 10:36:51
    Property Name Property Value
    Detection Time 06/03/2013 10:36:52
    Secure Channel Error
    Semaphore Timeouts 0
    Semaphore Waiters 0
    Semaphore Acquires 0
    Semaphore Holders 0
    Average Semaphore Hold Time 06/03/2013 10:36:52
    State 0
    ErrorNumber 
    ErrorDescription Could not Read Category Index: 1848.
    -----------------

    Date and Time: 03/06/2013 08:57:10
    Property Name Property Value
    Detection Time 06/03/2013 08:55:31
    Secure Channel _Total
    Semaphore Timeouts 0
    Semaphore Waiters 0
    Semaphore Acquires 0
    Semaphore Holders 0
    Average Semaphore Hold Time 0
    State 1
    ErrorNumber 0
    ErrorDescription 

    Monday, June 03, 2013 10:23 AM
  • Hello,

    Actually, State 1 is the 'Healthy' state. Try to open the health explorer and execute the 'List All Max Concurrent API Performance Data' diag task from the state change events tab. What do you see?


    http://OpsMgr.ru/

    Monday, June 03, 2013 2:50 PM
    Moderator
  • Thank you Alexey, the alert context with all values 0 (except the state parameter) is the closed state of the alert with "ErrorDescription Could not Read Category Index: 1848."

    I will disable the monitor for the servers affected.

    Regards

    Monday, June 03, 2013 4:19 PM
  • What do you see in the alert context?

    http://OpsMgr.ru/

    Ok, I understand what you were asking now. I'm getting the following alert context on a regular basis with the details listed below it:

    Date and Time: 6/3/2013 9:56:28 AM 

    Property Name Property Value 

    Detection Time 06/03/2013 09:56:28 

    Secure Channel Error 

    Semaphore Timeouts 0 

    Semaphore Waiters 0 

    Semaphore Acquires 0 

    Semaphore Holders 0 

    Average Semaphore Hold Time 06/03/2013 09:56:28 

    State 0 

    ErrorNumber  

    ErrorDescription Could not Read Category Index: 1848. 

    Task History: 
      
      Diagnostic task ran successfully:  List All Max Concurrent API Performance Data 
      Scheduled Time:  6/3/2013 6:41 AM 
      Start Time:  6/3/2013 6:41 AM 
      Finish Time:  6/3/2013 6:41 AM 
      Submitted by:   
      Diagnostic Output:  Date and Time: 6/3/2013 6:34:38 AM 
    Property Name Property Value 
    Detection Time 06/03/2013 06:41:48 
    Secure Channel server.xxx.xxx.xx 
    Semaphore Timeouts 0 
    Semaphore Waiters 0 
    Semaphore Acquires 4000 
    Semaphore Holders 0 
    Average Semaphore Hold Time 0 
    State 1 
    ErrorNumber 0 
    ErrorDescription 

     Thanks again for your help.

    Monday, June 03, 2013 4:26 PM
  • When I run the 'List all Max Concurrent API Performance Data' task, I get the following:

    No recoveries are available for this diagnostic 
      Diagnostic task ran successfully:  List All Max Concurrent API Performance Data 
      Scheduled Time:  6/3/2013 10:53 AM 
      Start Time:  6/3/2013 10:53 AM 
      Finish Time:  6/3/2013 10:53 AM 
      Submitted by:  WSD\justin.gray 
      Diagnostic Output:  Date and Time: 6/3/2013 10:46:03 AM 
    Property Name Property Value 
    Detection Time 06/03/2013 10:53:12 
    Secure Channel server.xxx.xx.xx 
    Semaphore Timeouts 0 
    Semaphore Waiters 0 
    Semaphore Acquires 4625 
    Semaphore Holders 0 
    Average Semaphore Hold Time 0 
    State 1 
    ErrorNumber 0 
    ErrorDescription  

     
        The diagnostic task ran successfully. You may attempt a recovery by passing this output information to the recovery task(s) listed below. 
        Recovery Tasks: 
         
        No recoveries are available for this diagnostic 

    Monday, June 03, 2013 4:55 PM
  • Hi Justin,

    >Secure Channel Error

    >ErrorDescription Could not Read Category Index: 1848.

    The monitor should be disabled for this server(s) due to a bug.


    http://OpsMgr.ru/

    • Marked as answer by j-gray Monday, June 03, 2013 8:16 PM
    Monday, June 03, 2013 5:51 PM
    Moderator
  • Thanks for the insight.  Ran similar tests and found my servers seemingly unaffected, but still getting the alert.  Disabling for now.
    Tuesday, June 25, 2013 6:40 PM
  • Greetings all,

    Sorry to re-open an apparently closed topic, but as I have the exact same issue, I would like to provide a little more info and also be sure that I can discard the alerts I have.

    So, since the Windows Server MP upgrade, I'm getting "Max Concurrent API reached" alerts on a few servers.

    I first read the KB2688798 (about this very issue) and tried to do the calculations myself from the results I get in PerfMon.  But the formula is somehow biased:
    semaphore_acquires + semaphore_time-outs) * average_semaphore_hold_time / time_collection_length = New_MaxConcurrentApi_setting
    This fails to give you a proper result, when like me, your average_semaphore_hold_time equals 0.  Plus, I noticed that the final result is very dependant on time_collection_length, and I noticed that whatever was the collection length (I tried 15 seconds, then I treid 5 minutes), the values of semaphore_acquires and semaphore_time-outs where exactly the same.

    Then I found this technet article by Tim Springston (AD team), in which there's a link to download a very useful PowerShell script which does all the calculations advised in the KB, but somehow manages to give you a final advise : "Problem detected" or not.

    I ran the Powershell script on all the servers I could, and every result was (in the Powershell output) : Problem detected: False

    Then I found this post here, and I (successfully) ran the "List All Max Concurrent API Performance Data" on the faulting servers. Here are the relevant values:
    Secure Channel : Error
    All semaphores values : 0
    Error Number : [empty]
    ErrorDescription : Category does not exist

    Then I realized that almost all my faulting servers are in specific LAN configs. Some are in a DMZ, some are in their own VLAN with specific firewall settings.  Normally all these servers have the usual SCOM ports open to allow correct communications between MS and agents, but here's the question: doesn't that Management Pack rule use different network settings than the ones SCOM is supposed and/or allowed to use?

    Now the mystery about the MaxConcurrentAPI isn't totally solved yet, as their remains one "regular" SQL server, which has some heavy load for sure, but as heavy as many others which don't give any problem in that MaxApi regard.

    The List All Max Concurrent API Performance Data on this server gives this:

    Secure Channel : _Total
    Semaphore Timeouts : 92
    Semaphore Waiters : 0
    Semaphore Acquires : 3152686
    Semaphore Holders : 0
    Average Semaphore Hold Time : 0
    State: 0
    ErrorNumber: 0

    ErrorDescription : [empty]

    So here, no apparent error. The task ran fine.  But then again, the result of the PowerShell script is "Problem Detected : False". Here is the complete output (relevant values) :

    Problem Detected                                             : False
    Current Effective MaxConcurrentApi Setting                   : 2
    Suggested MaxConcurrentApi Setting (may be same as current)  : 2
    Current Threads in Use (Semaphore Holders)                   : 0
    Clients Currently Waiting (Semaphore Waiters)                : 0
    Cumulative Client Timeouts (Semaphore Timeouts)              : 92
    Cumulative MaxConcurrentApi Thread Uses (Semaphore Acquires) : 3152686

    Same numbers as the State change task, but "problem : false".

    I don't know who to trust : the SCOM alert or the PowerShell from the AD team?


    Bix





    • Edited by Bixessss Wednesday, July 10, 2013 8:43 AM perfmon, not procmon :)
    Wednesday, July 10, 2013 8:27 AM
  • Hi,

    we'll post all details and resolutions we'll found in the SCC ResearchThis KB: http://www.systemcentercentral.com/research-this-kb-max-concurrent-api-reached-alert/

    (7/23/2013: this article is incomplete now and will be updated shortly). 

    "Then I realized that almost all my faulting servers are in specific LAN configs. Some are in a DMZ, some are in their own VLAN with specific firewall settings.  Normally all these servers have the usual SCOM ports open to allow correct communications between MS and agents, but here's the question: doesn't that Management Pack rule use different network settings than the ones SCOM is supposed and/or allowed to use?"

    It has nothing to do with network settings but with a domain membership. There is no Netlogon counters on workgroup machines.  Is this your case?


    http://OpsMgr.ru/

    Tuesday, July 23, 2013 4:04 AM
    Moderator
  • Sorry for late answer, coming back from holidays.

    Thanks a lot for your useful link to he systemcentral.com topic, at least I get some relevant information.

    Like you said here in your answer and like it is said in the SC article, computers in workgroups don't have the necessary counters and the rule is failing. 

    I have a lot of matter on my hands and I grow tired of these alerts, so for now I will follow the instruction from the article :

    "Resolution: Created an oveeride to disable both of these monitors as they do not appear to work"

    And I will check the article on SC for further updates, and don't hesitate to reply to this topic when the problem is corrected in a future new version of the MP.

    Thanks again Alexey.


    Bix


    • Edited by Bixessss Friday, August 09, 2013 1:47 PM layout
    Friday, August 09, 2013 1:46 PM