locked
SCOM Management Packs not getting applied for Windows 2016 DC servers RRS feed

  • Question

  • We have installed SCOM 2016 on 4 of our Windows 2016 DC servers. Out of which 2 are working fine. We have two servers for which the management packs are not getting applied. Except "Agent2012.HealthState" management pack none of the other 5 packs are getting applied. Due to this our server and AD monitoring is failing. 

    The servers were working fine before DC Promote and IP change. But these two are having troubles after DC Promote and IP change and other two are just working fine. All 4 servers underwent DC Promote and IP Change. We even tried uninstalling SCOM from the server, then from the console and then trying a reinstall of the agent on the server again. Still no luck.

    Below packs are applied on the working server

    Microsoft.Windows.FileServer.SMB.2016
    Agent2012.HealthState
    Microsoft.Windows.Server.20xx
    WinOS.Extensions
    Microsoft.Windows.FileServer.DFSN.2016

    And below is the only pack getting applied on the server with issue

    Agent2012.HealthState

    Any help would be highly appreciated. 

    Wednesday, May 27, 2020 9:24 AM

Answers

  • Yes but the management packs that you are specifically looking for, are they there on the problematic servers?


    The issue has been fixed finally. All I had to do was install OOMADs.msi package and restart the SCOM services. :) . Monitoring is working fine now. 

    • Marked as answer by PK.Pradeep Friday, June 5, 2020 7:17 PM
    • Edited by PK.Pradeep Friday, June 5, 2020 7:18 PM https://social.technet.microsoft.com/Forums/systemcenter/en-US/5a64e774-3f14-4ecc-93ea-f36db59a4b07/adding-the-active-directory-helper-object-oomads-to-dc-servers?forum=operationsmanagerauthoring
    Friday, June 5, 2020 7:16 PM

All replies

  • Hi,

    You might want to take a look at the following blog post:

    Deploying SCOM 2016 Agents to Domain controllers – some assembly required
    https://kevinholman.com/2016/11/04/deploying-scom-2016-agents-to-domain-controllers-some-assembly-required/

    Check what the Operations Manager event logs are saying on the non-working Domain Controllers and verify if you see any of the events mentioned in the above blog post.

    Best regards,
    Leon


    Blog: https://thesystemcenterblog.com LinkedIn:

    Wednesday, May 27, 2020 9:31 AM
  • Thanks Leon, will check and update.
    Wednesday, May 27, 2020 9:37 AM
  • Also check for events indicating a lack of resource or memory such as (but not limited to) Id 2023 in Operations Manager event log, I've had this issue on some domain controllers since I deployed the new ADDS MP.
    Wednesday, May 27, 2020 9:37 AM
  • I had that issue. SYSTEM account was showing as denied, so I allowed as shown in the blog. But still the issue was the same, so when I compared it with the working ones, NT AUTHORITY\Authenticated Users was the only account mentioned under Allowed and NT AUTHORITY\SYSTEM was neither allowed nor denied. So I removed the account using the command HSLockdown.exe /R “NT AUTHORITY\SYSTEM”, but still the same issue. Doing HSLockdown.exe /A “NT AUTHORITY\SYSTEM” helped in changing the state from RED to GREEN in SCOM console, but it did not help with fixing monitoring or applying management packs. And HSLockdown.exe /R “NT AUTHORITY\SYSTEM doesn't seem to have helped either. 

    Is there a way to push management packs by force or is there a command to run on the DC to check what management packs are applied to that server?

    Wednesday, May 27, 2020 9:42 AM
  • Also check for events indicating a lack of resource or memory such as (but not limited to) Id 2023 in Operations Manager event log, I've had this issue on some domain controllers since I deployed the new ADDS MP.
    Well, the server resources are aplenty. And neither ADDS MP or any other management packs are getting applied.
    Wednesday, May 27, 2020 9:45 AM
  • Clear the HealthService queue and config (manually). This is done by stopping the Microsoft Monitoring Agent service, then deleting the "Health Service State" folder within the C:\Program Files\Microsoft Monitoring Agent\Agent folder.

    The agent will then contact the management server to request config, receive config, download the appropriate management packs, apply them, run the discoveries, send up discovery data.


    Blog: https://thesystemcenterblog.com LinkedIn:


    • Edited by Leon Laude Wednesday, May 27, 2020 9:47 AM
    Wednesday, May 27, 2020 9:47 AM
  • Also check for events indicating a lack of resource or memory such as (but not limited to) Id 2023 in Operations Manager event log, I've had this issue on some domain controllers since I deployed the new ADDS MP.

    Well, the server resources are aplenty. And neither ADDS MP or any other management packs are getting applied.
    The server may have plenty of resource but still the agent might not be allowed to use them as it needs 
    Also it's not very clear what you mean by "getting applied". What's the issue exactly? No instance discovered at all? Instances discovered but staying unmonitored (blank)?
    • Edited by CyrAz Wednesday, May 27, 2020 9:49 AM
    Wednesday, May 27, 2020 9:48 AM
  • Clear the HealthService queue and config (manually). This is done by stopping the Microsoft Monitoring Agent service, then deleting the "Health Service State" folder within the C:\Program Files\Microsoft Monitoring Agent\Agent folder.

    The agent will then contact the management server to request config, receive config, download the appropriate management packs, apply them, run the discoveries, send up discovery data.




    @Leon - Thanks Leon, will try that. 
    Wednesday, May 27, 2020 9:49 AM
  • Also check for events indicating a lack of resource or memory such as (but not limited to) Id 2023 in Operations Manager event log, I've had this issue on some domain controllers since I deployed the new ADDS MP.

    Well, the server resources are aplenty. And neither ADDS MP or any other management packs are getting applied.

    The server may have plenty of resource but still the agent might not be allowed to use them as it needs 
    Also it's not very clear what you mean by "getting applied". What's the issue exactly? No instance discovered at all? Instances discovered but staying unmonitored (blank)?
    By applied in the sense the management packs are not getting assigned to the server. Except for Agent2012.HealthState pack. So the issue is, SCOM is installed on the DC, shows green in the SCOM console, shows all OK and Active in the SCOM reports, but when checked for the management packs assigned to the servers it shows only the above pack and not the others as shown in the original post. Due to this the alerts are not landing on our ticketing tool which is an issue as the servers are in production and currently not being monitored.
    Wednesday, May 27, 2020 10:40 AM
  • Clear the HealthService queue and config (manually). This is done by stopping the Microsoft Monitoring Agent service, then deleting the "Health Service State" folder within the C:\Program Files\Microsoft Monitoring Agent\Agent folder.

    The agent will then contact the management server to request config, receive config, download the appropriate management packs, apply them, run the discoveries, send up discovery data.




    @Leon - Thanks Leon, will try that. 
    I have done as suggested, but not sure how long will it take to know if it has helped to fix the issue. As requested earlier, are you aware of any commands that can be run from the DC to check if the management packs are applied or any scripts that can be run to perform healthcheck of the agent and to make sure all is well?
    Wednesday, May 27, 2020 10:43 AM
  • There are no commands for this, the only manual way is the method I described in my previous reply.

    The best way to check which management packs have been received/configured is by checking the Operations Manager event log.

    The servers were working fine before DC Promote and IP change. But these two are having troubles after DC Promote and IP change and other two are just working fine. All 4 servers underwent DC Promote and IP Change. We even tried uninstalling SCOM from the server, then from the console and then trying a reinstall of the agent on the server again. Still no luck.

    Try:

    1. Uninstalling the SCOM agents on the problematic Domain Controllers (DCs).

    2. Make sure the folders and registry entries have also been deleted.

    3. Reboot the problematic DCs.

    4. Clear the cache of the SCOM management servers (How and When to Clear the Cache)

    5. Clear the cache of the Operations Console (How and When to Clear the Cache)

    6. Reinstall the SCOM agents on the problematic DCs.

    7. Give it up to an hour and check the status.

    8. If the issue is still the same, check the Operations Manager event logs on the problematic DCs for any warnings/errors.


    Blog: https://thesystemcenterblog.com LinkedIn:

    Wednesday, May 27, 2020 11:22 AM
  • There are no commands for this, the only manual way is the method I described in my previous reply.

    The best way to check which management packs have been received/configured is by checking the Operations Manager event log.

    The servers were working fine before DC Promote and IP change. But these two are having troubles after DC Promote and IP change and other two are just working fine. All 4 servers underwent DC Promote and IP Change. We even tried uninstalling SCOM from the server, then from the console and then trying a reinstall of the agent on the server again. Still no luck.

    Try:

    1. Uninstalling the SCOM agents on the problematic Domain Controllers (DCs).

    2. Make sure the folders and registry entries have also been deleted.

    3. Reboot the problematic DCs.

    4. Clear the cache of the SCOM management servers 

    5. Clear the cache of the Operations Console 

    6. Reinstall the SCOM agents on the problematic DCs.

    7. Give it up to an hour and check the status.

    8. If the issue is still the same, check the Operations Manager event logs on the problematic DCs for any warnings/errors.



    Thanks Leon, will try this as well.
    Wednesday, May 27, 2020 12:20 PM
  • There are no commands for this, the only manual way is the method I described in my previous reply.

    The best way to check which management packs have been received/configured is by checking the Operations Manager event log.

    The servers were working fine before DC Promote and IP change. But these two are having troubles after DC Promote and IP change and other two are just working fine. All 4 servers underwent DC Promote and IP Change. We even tried uninstalling SCOM from the server, then from the console and then trying a reinstall of the agent on the server again. Still no luck.

    Try:

    1. Uninstalling the SCOM agents on the problematic Domain Controllers (DCs).

    2. Make sure the folders and registry entries have also been deleted.

    3. Reboot the problematic DCs.

    4. Clear the cache of the SCOM management servers 

    5. Clear the cache of the Operations Console 

    6. Reinstall the SCOM agents on the problematic DCs.

    7. Give it up to an hour and check the status.

    8. If the issue is still the same, check the Operations Manager event logs on the problematic DCs for any warnings/errors.



    Thanks Leon, will try this as well.
    Tried that today alongside SCOM team as I do not have console access and it did not help.
    Thursday, May 28, 2020 3:20 PM
  • 8. If the issue is still the same, check the Operations Manager event logs on the problematic DCs for any warnings/errors.


    Blog: https://thesystemcenterblog.com LinkedIn:

    Thursday, May 28, 2020 3:21 PM
  • 8. If the issue is still the same, check the Operations Manager event logs on the problematic DCs for any warnings/errors.



    Tried that as well, unfortunately nothing seems conclusive. Even the internal SCOM team is clueless. Not sure what else we should try to fix the monitoring issue.
    Thursday, May 28, 2020 6:10 PM
  • Any firewalls / hardenings or other security measures that may prevent SCOM?

    Blog: https://thesystemcenterblog.com LinkedIn:

    Thursday, May 28, 2020 6:15 PM
  • Any firewalls / hardenings or other security measures that may prevent SCOM?


    SCOM team did check that as well, they said all is ok. Agent is communicating just fine and is active it seems. But for some reason monitoring doesn't work and the management packs are not assigned.

    I was able to ping and telnet over port 5723 to the gateway server as well.

    If I export the reg keys from a working server and import it to the problematic server, will that work? If so, what would be order of action to perform that? 


    • Edited by PK.Pradeep Friday, May 29, 2020 9:11 AM
    Friday, May 29, 2020 9:10 AM
  • I'll ask again : what exactly do you mean by "management pack not assigned"?
    Friday, May 29, 2020 9:11 AM
  • I'll ask again : what exactly do you mean by "management pack not assigned"?

    I am not sure if it gets assigned, or applied or alloted. But below are the management packs reporting for a server that is working fine.


    Atos.AS.Global.Microsoft.Windows.Server.AD.2016
    Atos.AS.Global.Microsoft.Windows.FileServer.SMB.2016
    Atos.SD.Global.Agent2012.HealthState
    Atos.AS.Global.Microsoft.Windows.Server.20xx
    Atos.AS.Global.WinOS.Extensions
    Atos.AS.Global.Microsoft.Windows.FileServer.DFSN.2016

    And the below is the only management pack I could see in the report for the problematic DC

    Atos.SD.Global.Agent2012.HealthState

    I am not a SCOM expert so I am not sure what these names mean.

    Friday, May 29, 2020 9:17 AM
  • What report? How are you checking this?

    All of the mentioned management packs appear to be custom authored management packs.


    Blog: https://thesystemcenterblog.com LinkedIn:

    Friday, May 29, 2020 9:21 AM
  • What report? How are you checking this?

    All of the mentioned management packs appear to be custom authored management packs.



    These are from the internal APS report that SCOM team generates every 24 hours to monitor server health and status stuff. We fetch this report from a SP link. And yes, these are customized packs from that of SCOM Management Packs that Microsoft releases, as per the company needs . Each company customize these packs as per their environment and requirement.
    Friday, May 29, 2020 9:23 AM
  • Edit: Apparently my question didn't show up for some reason, I wanted to ask how are you generating these reports? From where are they gathering the information about the applied management packs?

    Check the "Management Packs" folder which is located within the Microsoft Monitoring Agent installation folder on your problematic Domain Controllers to verify which management packs have been applied:

    • C:\Program Files\Microsoft Monitoring Agent\Agent\Health Service State\Management Packs


    Blog: https://thesystemcenterblog.com LinkedIn:


    • Edited by Leon Laude Friday, May 29, 2020 10:49 AM
    Friday, May 29, 2020 9:46 AM
  • What report? How are you checking this?

    All of the mentioned management packs appear to be custom authored management packs.



    These are from the internal APS report that SCOM team generates every 24 hours to monitor server health and status stuff. We fetch this report from a SP link. And yes, these are customized packs from that of SCOM Management Packs that Microsoft releases, as per the company needs . Each company customize these packs as per their environment and requirement.
    Can you ask them on what is based that report? How its info is gathered?
    Friday, May 29, 2020 10:12 AM
  • Edit: Apparently my question didn't show up for some reason, I wanted to ask how are you generating these reports? From where are they gathering the information about the applied management packs?

    Check the "Management Packs" folder which is located within the Microsoft Monitoring Agent installation folder on your problematic Domain Controllers to verify which management packs have been applied:

    • C:\Program Files\Microsoft Monitoring Agent\Agent\Health Service State\Management Packs




    They say they are running scripts on the Management server against the agents to fetch these reports it seems.
    Friday, May 29, 2020 11:10 AM
  • Yes but how and from exactly where is this data coming from?

    Did you check the Management Packs folder to verify? This would tell us whether the report is wrong or not.


    Blog: https://thesystemcenterblog.com LinkedIn:

    Friday, May 29, 2020 11:16 AM
  • Edit: Apparently my question didn't show up for some reason, I wanted to ask how are you generating these reports? From where are they gathering the information about the applied management packs?

    Check the "Management Packs" folder which is located within the Microsoft Monitoring Agent installation folder on your problematic Domain Controllers to verify which management packs have been applied:

    • C:\Program Files\Microsoft Monitoring Agent\Agent\Health Service State\Management Packs




    There are 427 packs on the working server and 360 on the problematic server.
    Friday, May 29, 2020 11:31 AM
  • Yes but the management packs that you are specifically looking for, are they there on the problematic servers?

    Blog: https://thesystemcenterblog.com LinkedIn:

    Friday, May 29, 2020 11:36 AM
  • Yes but the management packs that you are specifically looking for, are they there on the problematic servers?


    No they are not. They were not present on the problematic server. I tried copying and pasting them manually from a working server to the problematic server and tried triggering the alert for monitoring check, it doesn't seem to help.
    Friday, May 29, 2020 12:32 PM
  • Hi,

    Have the same query with CryAz, what does it mean when we say the management packs are not applied?

    After the operations manager agent is installed, it will pull management packs from the server and store it in the following path

    Program Files\Microsoft Monitoring Agent\Agent\Health Service State\Management Packs

    For example, our lab domain controller is 10.1.1.10 and we can check the location to see if the management packs are there, as shown in below sample.




    Hope the above information helps.

    Regards,

    Alex Zhu
    -----------------------------------------------
    Please remember to mark the replies as answers if they help.
    If you have feedback for TechNet Subscriber Support, contact tnmff@microsoft.com.
    Monday, June 1, 2020 8:34 AM
  • Hi,

    Have the same query with CryAz, what does it mean when we say the management packs are not applied?

    After the operations manager agent is installed, it will pull management packs from the server and store it in the following path

    Program Files\Microsoft Monitoring Agent\Agent\Health Service State\Management Packs

    For example, our lab domain controller is 10.1.1.10 and we can check the location to see if the management packs are there, as shown in below sample.


    We checked this already and there are 427 packs on the working server and 360 on the problematic server. So there are packs that are missing and some of them are the ones that are configured to be used by our SCOM agent.

    Thursday, June 4, 2020 3:56 PM
  • After days of troubleshooting, the SCOM team says the issue is with their gateway server in that location and they need to fix it which is another time consuming tasks. Waiting to hear back from them.
    Thursday, June 4, 2020 3:57 PM
  • Yes but the management packs that you are specifically looking for, are they there on the problematic servers?


    The issue has been fixed finally. All I had to do was install OOMADs.msi package and restart the SCOM services. :) . Monitoring is working fine now. 

    • Marked as answer by PK.Pradeep Friday, June 5, 2020 7:17 PM
    • Edited by PK.Pradeep Friday, June 5, 2020 7:18 PM https://social.technet.microsoft.com/Forums/systemcenter/en-US/5a64e774-3f14-4ecc-93ea-f36db59a4b07/adding-the-active-directory-helper-object-oomads-to-dc-servers?forum=operationsmanagerauthoring
    Friday, June 5, 2020 7:16 PM