none
Empty Target Group for Application Component

    Question

  • I set up a .Net APM item in SCOM the other day which contained two components site (A) and site (B). I targeted the item to a Group containing two IIS servers, left config items as default, and enabled Client side monitoring as well.

    After about five minutes site (A) became monitored and site (B) remained not monitored. There were warning events generated that IIS needed to be restarted on the IIS Server containing site (A) but nothing about the IIS server with site (B). There was also a warning alert under .Net Monitoring stating the Empty Target Group for Application Component that the targeted group did not contain any servers hosting the applications.

    We restarted the IIS Service on both IIS boxes and still nothing. We ran the Check Client-Side Monitoring Compatibility and both sites (A) and (B) came back successful. Restarted the SCOM Management service and SCOM APM service on the IIS server containing site (B), but still site (B) remains Not Monitored.

    We've added a couple of extra applications in to APM and the same results happen where site (A) works and site (B) is not monitored.

    Any suggestions?

    Friday, March 15, 2013 4:17 PM

Answers

  • Coming back to this thread in case other stumble into it. We had this issue also reported and escalated by a customer thru Support, and we were able to investigate it. There was a bug that caused the error when the DisplayName of the windows computer (as discovered by OpsMgr) is lower case when some of the other fields that also refer to its NetBIOS name were upper case...

    We don't know exactly when this happens (=it might depend on different windows versions since discovery uses information on the local agent, thru registry, WMI and also LDAP queries in Active Diretcory...). In most cases, the NetBIOS portion of your computer names in OM will be UPPER-case and you won't get into this state. That is why we didn't catch this bug earlier.

    Anyway the good news is that we have fixed this in System Center 2012 R2 as well as back-ported the fix for System Center 2012 SP1 in the Update Rollup 4 - http://support.microsoft.com/kb/2879276/en-us

    Wednesday, October 23, 2013 9:39 PM
    Owner

All replies

  • The error means that the business logic on the OpsMgr system can't find an instance of the application component on any or some of the servers in the group. By how you describe it, it seems that your application components are not all present on all those machines - hence the error.

    If the component "lives" only on a given server, that's the only place where it will be monitored. In most cases you do not need to use group scoping. Group scoping is only useful when:

    1) you need to restrict which part of our farm gets monitored (i.e. only monitor 1 instance out of X of a given application component)

    2) you want to distinguish "environments" (such as "production" and "staging") and you know which servers represent one environment or the other one... while OpsMgr would think those components are really all the same.

    Try starting simple: configure a template for a SINGLE component and don't specify ANY group scoping option. If you HAVE to specify a group for the reason above, use a new group that you create for the purpose, and that contains only the "windows computer" object for the server that has the component. Wait for config to be re-calculated and see if the alert comes back. Then proceed with the other component and machine with a separate template.

    HTH,

    Sunday, March 17, 2013 3:59 PM
    Owner
  • Under the IIS 7.0 ASP.NET Web Application Inventory the site (B) is discovered in that list on the IIS Server that is referenced in the group. The group that I am using for scoping purposes only contained the two windows computer objects for the two IIS servers these sites reside on.

    Following your instructions I created a new object with only a component from site (B) and I did not specify any group scoping option but still got the exact same error in this scenario as well:

    The application components site(B) - Production included in Monitor - Production are scoped by target
    groups that do not contain any servers hosting the applications. These
    application components will not be monitored until the targeted group is updated
    to include servers that host the applications.

    Sunday, March 17, 2013 9:06 PM
  • Just to confirm:

    1) the NEW application group (template instance) contains ONLY ONE component, and NO Group scoping?

    2) the OLD template does NOT contain the same component anymore?

    Does the component surely and really exist on IIS7 / Windows Server 2008 on that box? We saw a case with MSIT where the machine had been FORMATTED and RE-INSTALLED with a new operating system (Windows Server 2012 / IIS8) and the objects were being *duplicate* (old and new version) and this was confusing the business logic.

    What's the NAME (yes the actual name) of the component? Just curious if we choke on something there.

    Otherwise I am a big at loss as to what to suggest on a forum - I would need to look at verbose trace logs from the Management Server(s)... maybe easier done thru official support channel to understand if it is an environment-specific issue or a bug.

    Sunday, March 17, 2013 10:26 PM
    Owner
  • 1) Correct I created a brand new application group (template instance) that only contains site (B) and nothing else and I did not target any server group it is blank.

    2) I created a new application group from an entirely different site which have different components so not using the same one as one already created.

    Yes the component without any doubt resides on the IIS7 / Windows Server 2008 R2 box. When I look in IIS 7.0 ASP.NET Web Application Inventory it shows Site (B)(IIS Application name)  /LM/W3SVC/9/ROOT (Application Virtual Root) and SERVER.DOMAIN.COM;WESVC/9 (Path). There has been no formatting or new OS installed and the item resides there.

    The actual name for the brand new instance is:

    B - Loan Modification System (LMS)

    I don't feel this would be the cause unless the "B" causes issues, because the component with the same name but A - Loan Modification System (LMS) works with no issues. As well as all other instances of A - Name, only the B - Name are not being monitored.

    Seems like this is an issue I will need to bring up with our TAM, thank you for the suggestions though. It's very strange situation to us seeing how seamlessly it was for the A - Name components to be monitored.

    Sunday, March 17, 2013 10:37 PM
  • Hold on - are you saying that both "A" and "B" are both "Loan Modification System (LMS)" - basically they have the same name?

    Well in that case they ARE the same component, from OpsMgr's point of view. Even if you see al the instances of a given component in the inventory view, the template only shows you a DISTINCT (UNIQUE) view of all components. I can have  a given application running on N number of servers in a web farm - in the wizard I only see ONE entry for this (the *logical* component) not one for each instance (each one living on a different server).

    The first time you have selected the component, you are already aiming at targeting all of its instances, if you don't use group scoping. There should be no need to do it again from a second template run, IF NOT for the scenario of telling OpsMgr which machines are hosting the "production" version of this, and which ones are hosting the "staging" version of it (or some other TAG/environment).

    If the names are instead ALREADY *unique* (i.e. they contain "A -" and "B - " in the name) then you'll see them as distinct ones in the wizard/selection.

    I can't tell if you are already distinguishing them by environment based on the naming convention (would be a good thing - means you never need to specify any group) or if you need to and are trying to have OpsMgr do the saparation between enviornments (or versions) of the application based on groups you define.

    Sunday, March 17, 2013 10:51 PM
    Owner
  • So A - Loan Modification System (LMS) resides on Server1, and B - Loan Modification System (LMS) resides on Server2 obviously for redundancy. When I create the LMS APM Template I see both A - Loan Modification System (LMS) and B - Loan Modification System (LMS). So I pick both instances and after a few minutes only A - Loan Modification System (LMS) is being monitored in our Production SCOM instance.

    If I do the same scenario in our test SCOM instance with the same exact settings, both A and B instance are being monitored. Since they are redundant end-users can hit either Server1 or Server2 so for Client Side monitoring you would want both sites monitored.

    Sunday, March 17, 2013 11:00 PM
  • I see, so basically they ARE the same site, but are given different names. Is there a reason why the web applications/sites are given different names since they are essentially the same app, and both are "production"? I have never seen this done this way (renaming individual instances of the same thing); can't you simply tell them apart by the machine name they live on (which lives in the "Path" of the object thru the hosting relationship)? That would essentially make them the "same" component, and you would only have to select it once, with NO group scoping, and both would (should) be monitored.

    In any case, given that now they have different names, you should indeed be able to add them both to the SAME template, again without group scoping, and it should propagate configuration wherever it sees either one or the other (that is, to both boxes).

    Side question (haven't had you check that): do you find that both "System Center Management" and "System Center Management APM" are installed on BOTH machines? (i.e. also on the one that doesn't work)

    Also, are they reporting to different management servers (maybe in different MS pools) per chance ?

    Sunday, March 17, 2013 11:34 PM
    Owner
  • Right they are the same exact sites, they have different config files not a network share, but essentially the config files are identical as well. I can't speak to the reasoning behind naming one (A) and one (B) just something someone decided at some point I assume.

    I think you would still want to monitor both sites since they reside on different servers and you could potentially experience different problems i.e. one server utilizing more memory or processor causing the end-users connected to that server to see a different application experience.

    We are seeing the results you mentioned about being able to add them both to the same template in our Test environment, just can't get the same thing to reciprocate in production.

    Both System Center services are on both servers, and both are running. I even tried an uninstall/reinstall on the Server that wasn't working and it re-discovered all the (B) components again but same results.

    They both report to the same SCOM Management server, took a look at that as well.

    Monday, March 18, 2013 12:42 AM
  • Right. I don't think the issue is the agent... that seems to have discovered the object just fine. The error you are getting is from the Management Server which, for some reason, fails to "see" this component on that agent... I would try restarting the services (all three) on the management server, just in case... but ultimately would need to have a good look at a Verbose ETL tracing from that management server. Probably best handled thru support in this case, just for verbosity of the log itself... if you open a case please reference this thread, so the support engineer can have previous context and they can escalate to us in the product group if they need.
    Tuesday, March 19, 2013 12:16 AM
    Owner
  • We have a ticket open with Premier support on this. Will provide update when we have it.
    Tuesday, May 14, 2013 2:39 PM
  • Daniele, I may open another thread on this, but I am seeing the same behavior but in an even simpler setup in a lab: a single app living on a single 2008 r2 iis server. no target group or environment configured. still getting the "empty target group" warning.
    Saturday, May 18, 2013 3:20 AM
  • Do you find that both services "System Center Management" and "System Center Management APM" are installed on the agent?
    Monday, May 20, 2013 4:19 PM
    Owner
  • I see both services installed, but APM is disabled. I don't think we are going to use client-side monitoring though. the client side test showed a lot of critical errors. is that required?
    Tuesday, May 21, 2013 1:52 AM
  • Nothing is required :-) You can choose to use Client-side monitoring (in addition to server-side) or not. But let's not mix topics.

    The service being disabled is OK - it will be enabled once the configuration reaches the box, which has not done yet because you are getting that error on the server side. What that error means is that the server is not able to find that application (or the apm agent) on any agent machine (in the absence of group scoping)... I would think something is wrong with the template's configuration (i.e. has the site changed since it was configured?) and I would try to:

    1) remove that template altogether (also, is it the only one or do you have multiple ones that might have conflicting configuration?)

    2) wait; reset the monitor; wait to see it does not come back (restarting healthservices on servers and agents in this phase just in case to speed up the process)

    3) reconfigure

    Tuesday, May 21, 2013 5:06 PM
    Owner
  • I only have the one template. it's the first one I've stood up in my lab. I have deleted it and recreated it several times, sometimes with a target group, sometimes with an environment sometimes with one or the other, sometimes with neither... same error every time.

    Wednesday, May 22, 2013 5:07 PM
  • Right, I am not sure how many times you have tried and what sort of settings have now been added/removed to/from the system multiple times. I am not sure if the monitor was reset and the alert closed, if the services had been restarted, and if the right amount of time had been waited for in between all those tentatives... it can be very difficult to troubleshoot a moving target.

    As I wrote to Neal earlier, we would need to have a good look both at the MP itself, at the objects in the system, and at a Verbose ETL tracing from that management server to see what it says when it is trying to compute configuration...

    Wednesday, May 22, 2013 5:14 PM
    Owner
  • this may be something management-server-wide. to clarify on my setup : I have the same (barring slight tweaks due to development cycle) websites running in four environments: Dev, QA, UA, and Prod. UA and Prod are monitored by my "Prod" scom management group. Dev and QA are monitored by my QA/Lab management group -which is a single management server running sql locally.

    the same websites that give me the Empty Target Group error in my QA/Lab management group are *not* giving me the error in my Prod management group, so I guess something is not right with my QA/Lab management server.

    Tuesday, May 28, 2013 9:06 PM
  • Coming back to this thread in case other stumble into it. We had this issue also reported and escalated by a customer thru Support, and we were able to investigate it. There was a bug that caused the error when the DisplayName of the windows computer (as discovered by OpsMgr) is lower case when some of the other fields that also refer to its NetBIOS name were upper case...

    We don't know exactly when this happens (=it might depend on different windows versions since discovery uses information on the local agent, thru registry, WMI and also LDAP queries in Active Diretcory...). In most cases, the NetBIOS portion of your computer names in OM will be UPPER-case and you won't get into this state. That is why we didn't catch this bug earlier.

    Anyway the good news is that we have fixed this in System Center 2012 R2 as well as back-ported the fix for System Center 2012 SP1 in the Update Rollup 4 - http://support.microsoft.com/kb/2879276/en-us

    Wednesday, October 23, 2013 9:39 PM
    Owner
  • Hi Daniele,

    I hope you see this post.

    Yesterday I met with the following problem: our customer set up .NET AMP monitoring for one of their application. Application itself exist on two servers. One is development and second is production. Template configuration is nothing special. They added only instance of APM ASP.NET Web Application Component, no group restrictions, default SS and no CS monitoring. IIS was also restarted.

    Application name is identical on both servers and both IIS 8.0 ASP.NET Web Application Inventory endpoints are discovered.

    The problem is that we see only one instance of APM Web Application on Overall Component Health view.

    Both agent are installed but APM is disabled. It’s look like he don’t get any configuration.

    Luckily, I found this thread and your last post. Following examination, I found that we have exactly that situation. The display name of the server where APM Web Application instance is not found is in lower-case and all other attribute values regarding name are in upper-case.

    So we install 2012 SP1 UR4, before it was 2012 SP1 UR3, reinstall agent on server but with no luck. The instance is still not found.

    Interesting is that after agent reinstall and discovery is pass the display name is again in lower-case.

    Another thing that I have to mention is that this server was initially 2003 and was replaced with new installed 2012 with same IP. The SCOM agent on 2003 was deleted in SCOM console and after some time installed on new server. APM has been implemented after that.

    I am interested in your proposal how to move forward. Do we need open a case or is anything else that we can check or try.

    I appreciate any information.

    Thanks in advanced.

    Regards,

    Ivan

    Tuesday, January 28, 2014 4:28 PM
  • We haven't changed the way those objects are discovered; those have been and are contained in various other management packs over the years, and depend on how names are written in windows, active directory, and I believe it also changes from version to version.

    The fix changes the way those strings are COMPARED in code, to see if they are the same or not even if they have different case.

    Assuming you have done the ‘basic’ checks mentioned at the beginning of this thread (i.e. the agent actually HAS the APM agent piece installed, no previous AVIcode leftover, etc….).

    Also, did you upgrade the MANAGEMENT PACKs that come with UR4 ? The fix is in one of the dependency download modules in there… if you only do Microsoft Update you don’t get the complete fix (MU only dumps updated MPs on the filesystem). You shouldn't need to reinstall the agent either - the fix is for management server/console.

    Once the UR4 and MPs are in place and all the management servers have been updated and restarted, then you should have the new modules with the fix loaded on the MS. At this point, when you run thru the configuration wizard/template AGAIN and apply/save the intent you have captured in the wizard, it is then that the computers/sites/applications (as seen from the MS) are evaluated and the configuration is produced. It is at that point that you can see if that machine is a match or not. You’d need to enable verbose tracing on the management server(s) and see what happens when the template is saved and config gets applied…it will tell if it finds the server as a possible target / hosting the app or not.

    If it still doesn't find it, it might be a different issue.

    • Proposed as answer by Ivan_Kovac Wednesday, January 29, 2014 9:10 AM
    Tuesday, January 28, 2014 5:13 PM
    Owner
  • Daniele,<o:p></o:p>

    <o:p> </o:p>

    Thanks for quick replay.<o:p></o:p>

    <o:p> </o:p>

    Yes, I have upgrade MPs which come with update. But I didn’t restart MSs because it is not explicitly required. However it would make sense to include management servers restart in to upgrade procedure. In the future I will perform that.<o:p></o:p>

    After restart the second APM Web Application instance was immediately discovered!<o:p></o:p>

    <o:p> </o:p>

    Many thanks for advice.<o:p></o:p>

    Regards,<o:p></o:p>

    Ivan<o:p></o:p>

    Wednesday, January 29, 2014 9:09 AM