locked
Exchange 2013 Monitoring Mailboxes RRS feed

  • Question

  • I'm running exchange 2013 and it seems to have problems with health mailboxes (monitoring mailboxes). I have 5 mailbox databases on this server, DB1, DB2, DB3, DB4 and DB5.

    I know there are 2 health mailboxes per mailbox database.

    The command `Get-Mailbox -monitoring | Get-MailBoxStatistics` shows that several of these have not been accessed since December, even though they are in mounted Databases and powershell shows they are healthy.

    There is also 2 health mailboxes that are missing from the below output because I got warninings saying these mailboxes have not been logged into yet.

    -----

    [PS] C:\windows\system32> Get-Mailbox -monitoring | Get-MailBoxStatistics

    DisplayName   : HealthMailbox98fb2dc7692341ad8a3325ea2b14bbcc
    ItemCount     : 17426
    LastLogonTime : 12/8/2012 3:28:04 PM
    Database      :  DB1

    DisplayName   : HealthMailboxe27d88df28ae4d53af620604a83aca4d
    ItemCount     : 12386
    LastLogonTime : 2/4/2013 3:29:46 PM
    Database      : DB1

    DisplayName   : HealthMailbox815f0ff077a342f7889a53ef38e40256
    ItemCount     : 291
    LastLogonTime : 12/8/2012 3:26:43 PM
    Database      : DB3

    DisplayName   : HealthMailboxbca2f3409f1d4ae99098f48b574fc36d
    ItemCount     : 579
    LastLogonTime : 12/8/2012 3:26:05 PM
    Database      : DB2

    DisplayName   : HealthMailbox2412e1a0e5d9415b8328f653b2e42efe
    ItemCount     : 209
    LastLogonTime : 2/3/2013 6:08:41 PM
    Database      : DB3

    DisplayName   : HealthMailbox594b739a129941e688eafee6bbdfece6
    ItemCount     : 209
    LastLogonTime : 2/3/2013 6:09:52 PM
    Database      : DB2

    DisplayName   : HealthMailboxc401cae2a70d4d659f5758908582406e
    ItemCount     : 210
    LastLogonTime : 2/4/2013 3:28:59 PM
    Database      : DB4

    DisplayName   : HealthMailbox4b81211d555c4accad1f61a98700382e
    ItemCount     : 212
    LastLogonTime : 2/3/2013 6:09:46 PM
    Database      : DB5

    Warning: The user hasn't logged onto mailbox `<AD Domain>/users/HealthMailbox2c62876ca1bb45849c0daf2ecee6d715`, so there is no data to return. After the user logs on, this warning will no longer appear.

    Warning: The user hasn't logged onto mailbox `<AD Domain>/users/HealthMailbox4399c592609a464d95c993ecee46f671`, so there is no data to return. After the user logs on, this warning will no longer appear.

    -----

    I am also getting a lot of queued messages in the inboundproxy.com queue that have the subject line "Undelieverable: Inbound Proxy Probe". I dont know how to get these to stop. I just have to clean out the queue once a day to prevent it from getting to large.

    The last thing I notice is that I get a lot of audit errors in the event log when the system is trying to login to these mailboxes. That might be what is causing the issue. Here is one of the messages:

    -----

    An account failed to log on.
    Subject:
    Security ID: SYSTEM
    Account Name: <computer name here>$
    Account Domain: <AD DOMAIN>
    Logon ID: 0x3E7

    Logon Type: 8
    Account For Which Logon Failed:
    Security ID: NULL SID
    Account Name: HealthMailboxbca2f3409f1d4ae99098f48b574fc36d@<domain.tld>
    Account Domain:

    Failure Information:
    Failure Reason: Unknown user name or bad password.
    Status: 0xC000006D
    Sub Status: 0xC0000064

    Process Information:
    Caller Process ID: 0x1265c
    Caller Process Name: C:\Windows\System32\inetsrv\w3wp.exe

    Network Information:
    Workstation Name: <computer name here>
    Source Network Address: ::1
    Source Port: 34602

    Detailed Authentication Information:
    Logon Process: Advapi  
    Authentication Package: Negotiate
    Transited Services: -
    Package Name (NTLM only): -
    Key Length: 0

    This event is generated when a logon request fails. It is generated on the computer where access was attempted.

    The Subject fields indicate the account on the local system which requested the logon. This is most commonly a service such as the Server service, or a local process such as Winlogon.exe or Services.exe.

    The Logon Type field indicates the kind of logon that was requested. The most common types are 2 (interactive) and 3 (network).

    The Process Information fields indicate which account and process on the system requested the logon.

    The Network Information fields indicate where a remote logon request originated. Workstation name is not always available and may be left blank in some cases.

    The authentication information fields provide detailed information about this specific logon request.
    - Transited services indicate which intermediate services have participated in this logon request.
    - Package name indicates which sub-protocol was used among the NTLM protocols.
    - Key length indicates the length of the generated session key. This will be 0 if no session key was requested.

    -----

    As you can see, the system can not login to to this health mailbox account. The one in my example one of the mailboxes that has not been logged into since december.

    Is there anywhere to troubleshoot these? or possibly rebuild them? The is a huge lack of documentation on these mailboxes from microsoft, I  can not find any commands to interact with these other then 'get-mailbox -monitoring'


    • Edited by Talmen Monday, February 4, 2013 9:47 PM
    Monday, February 4, 2013 9:46 PM

Answers

  • Ill will keep this in mind. I did not try that. I have two UPNs on our AD Tree and apparently the UPN that the health mailboxes were assigned to do not match the UPN exchange was using to login the healthmail box in with.

    I changed all the UPNs on the health mailbox accounts to our Domain's other UPN and that fixed it (this made the account's name+UPN match the account that the security event log showed exchange was attempting to authenticate as.

    I don't get health mailbox errors anymore, and this actually seems to have cleared up the RCP over HTTPS issues I was having as well. (constant outlook disconnects every 20 minutes for every client connected). I didn't even think the two issues were related.

    • Proposed as answer by Martina_Miskovic Thursday, March 21, 2013 10:21 PM
    • Marked as answer by Talmen Thursday, March 21, 2013 10:26 PM
    Thursday, March 21, 2013 10:00 PM

All replies

  • Hello

    Thank you for your question.

    I am trying to involve someone familiar with this topic to further look at this issue.


    Terence Yu

    TechNet Community Support

    Wednesday, February 6, 2013 6:30 AM
  • Hello,

    Did you find a sollution to this problem?

    Tuesday, March 19, 2013 10:30 AM
  • Is there anywhere to troubleshoot these? or possibly rebuild them? The is a huge lack of documentation on these mailboxes from microsoft, I  can not find any commands to interact with these other then 'get-mailbox -monitoring'


    Hi,
    What you can do is to delete all of them and when done, restart the service "Microsoft Exchange Health Manager" (restart-service MSExchangeHM).
    After a few minutes, you will see that the monitoring mailboxes you need will be recreated.

    Martina Miskovic

    Tuesday, March 19, 2013 10:43 AM
  • Ill will keep this in mind. I did not try that. I have two UPNs on our AD Tree and apparently the UPN that the health mailboxes were assigned to do not match the UPN exchange was using to login the healthmail box in with.

    I changed all the UPNs on the health mailbox accounts to our Domain's other UPN and that fixed it (this made the account's name+UPN match the account that the security event log showed exchange was attempting to authenticate as.

    I don't get health mailbox errors anymore, and this actually seems to have cleared up the RCP over HTTPS issues I was having as well. (constant outlook disconnects every 20 minutes for every client connected). I didn't even think the two issues were related.

    • Proposed as answer by Martina_Miskovic Thursday, March 21, 2013 10:21 PM
    • Marked as answer by Talmen Thursday, March 21, 2013 10:26 PM
    Thursday, March 21, 2013 10:00 PM
  • Hi Talmen,
    Thanks for Sharing!
    Interesting reading about the UPN...I will definitely keep that in mind if I see the errors you did.

    Martina Miskovic

    Thursday, March 21, 2013 10:22 PM
  • Martina,

       I've been experiencing some related issues with HealthMailbox permissions, with logs in the event viewer showing login failures, etc.  I tried removing the mailboxes as you suggested with the Remove-Mailbox cmdlet, but after restarting the Microsoft Exchange Health Manager service they have not yet reappeared nor have new users been created in Active Directory.  Is there another step that I need to run to get those to be recreated?  Thanks.

    Eric

    Tuesday, April 16, 2013 5:09 PM
  • Martina,

       I've been experiencing some related issues with HealthMailbox permissions, with logs in the event viewer showing login failures, etc.  I tried removing the mailboxes as you suggested with the Remove-Mailbox cmdlet, but after restarting the Microsoft Exchange Health Manager service they have not yet reappeared nor have new users been created in Active Directory.  Is there another step that I need to run to get those to be recreated?  Thanks.

    Eric


    Whenever I have (for testing purposes) removed the HealthMailboxes in ADUC and restarted the service MSExchangeHM, the mailboxes has been recreated every time.

    The above was done with RTM and I haven't tested this with CU1 yet,
    New in CU1 is that the HealthMailboxes is created in the Container domain.local/Microsoft Exchange System Objects/Monitoring. Have you checked there?

    Martina Miskovic

    Tuesday, April 16, 2013 5:15 PM
  • I don't see a "Monitoring" folder in that location. I am running CU1.  Would rerunning setup.exe and preparing AD possibly recreate them?  Or creating a new mailbox database and migrating everyone's mailboxes to that possibly fix it?  (I only have 10 users).
    Tuesday, April 16, 2013 5:23 PM
  • I don't see a "Monitoring" folder in that location. I am running CU1.  Would rerunning setup.exe and preparing AD possibly recreate them?  Or creating a new mailbox database and migrating everyone's mailboxes to that possibly fix it?  (I only have 10 users).

    Answer found in your own thread here

    Martina Miskovic

    Tuesday, April 16, 2013 7:01 PM
  • You can recreate the health mailboxes if nothing else works.

    http://howexchangeworks.com/2013/08/recreate-exchange-2013-health-mailboxes.html


    Rajith Enchiparambil | http://www.howexchangeworks.com |

    HowExchangeWorks.Com

    Thursday, August 8, 2013 3:37 PM
  • What if I disable the "MSExchangeHM" service? May I continue to use Exchange server without this functionality?

    Best regards,

    Michael Firsov

    Friday, August 9, 2013 6:01 AM
  • Yes you can.

    But you will be losing out the "managed availability". There will be errors relating to it in the event viewer. Why would you want to disable it?


    Rajith Enchiparambil | http://www.howexchangeworks.com |

    HowExchangeWorks.Com

    Friday, August 9, 2013 7:20 AM
  • Because I KNOW it can lead to problems and do NOT know whether it can ever help me or not.

    Here is one example of why I don't want it enabled:

    http://social.technet.microsoft.com/Forums/exchange/en-US/7ed96489-92dd-441a-93e0-3d805b807dc3/maildeliveryprobemaildeliveryprobecom-and-inboundproxyinboundproxycom-in-the-logs

    "

    In my opinion there's one more strangeness about Managed Availability in Exchange 2013:

    http://blogs.technet.com/b/exchange/archive/2012/09/21/lessons-from-the-datacenter-managed-availability.aspx

    "When something is unhealthy its first action is to attempt to recover that component. Managed Availability provides multi-stage recovery actions – the first attempt might be to restart the application pool, the second attempt might be to restart service, the third attempt might be to restart the server, and the final attempt may be to offline the server so that it no longer accepts traffic. If these attempts fail, managed availability then escalates the issue to a human through event log notification."

    ...am I getting it right that Managed Availability is not going to alert an administrator BEFORE restarting/taking the server offline...just AFTER and in case it was unable to fix the issue??????!!!!!

    "

    Friday, August 9, 2013 7:31 AM
  • Hi, I found in one of my enviroments this still did not work.

    I changed the startup account on the health monitoring service to my Admin account and restarted, within a few moments the mailboxes appeared, I change the account on the service back and everything is working as expected.

    See my blog for a quick instruction:

    http://gerhardwessels.wordpress.com/2014/01/30/exchange-2013-health-monitoring-mailboxes-missing/


    Gerhard Wessels


    • Edited by Gerhard_Wessels Thursday, January 30, 2014 7:30 AM Signature Wrong
    Thursday, January 30, 2014 7:29 AM
  • Yes it is related with UPN
    1.so i start install with domain in DC xxx.example.com.

    2. then create other mailboxdatabase + create user@xxx.example.com

    3. then change default domain to @example.com in ecp admin panel.

    4. now all users can't login with user@example.com, so i go to active directory and change account domain to @example.com
    5.  somewhere i did mistake, and to my created mailbox database HEALTH mailbox users can't login ( this appear in event logs.) in default mailboxdatabase all with logins was fine.
    6. i tray to change heatlmailboxes login domains in AD to xxx.example.com to example.com ad revers. so isuse sill hear they cant login.

    7. my help was to recreate health mailboxes like in this page
    http://it.gamerz-bg.com/index.php/failed-security-audits-4265-hmworker-exchange-2013/


    And issue gone. 

    Friday, March 7, 2014 1:32 PM
  • Hi,

    Everytime Exchange "Microsoft Exchange Health Manager" service restart a new health mailbox will be created. As said martina you can just delete the health mailboxes and restart the services. health mailboxes for each databases will be created automatically. Thanks.


    Regards, Riaz Javed Butt Consultant Microsoft Professional Services MCITP, MCITP (Exchange), MCSE: Messaging, MCITP Office 365

    Friday, March 7, 2014 5:33 PM
  • Martina, do you like sending people on a hunt through threads to find a solution?

    I have the same problem, no "Monitoring" folder in ADUC "Microsoft Exchange System Objects". I went 3 times through the thread and was not able to find it.

    It maybe convenient to reference to another thread, but it is not helpful when one has to go down several rabbit holes looking for that information and ending up in chains and loops.


    Gerhard Waterkamp ACSLA Inc.

    Thursday, May 29, 2014 7:12 PM
  • In AD Users and Computers, go to view and make sure advanced features is selected.  You should then be able to see the folder she describes in the tree.

    Friday, June 27, 2014 12:28 PM
  • Hi  

    I am looking for some documentation about  health mailboxes. 

    Is there some Microsoft article were it was documented, how they works, for what for they are need? 

    Friday, August 1, 2014 1:43 PM
  • "Because I KNOW it can lead to problems and do NOT know whether it can ever help me or
    not."

    > Absolute Right and NO this is not an obsolete Question. CLEAN up THE Health Crap now MS! provide REAL HELP.

    I have been doing Exchange since Version 5.X now for 15 years. I have been struggling with
    Exchange 2013/2016 now for months in complex VM LAB setups before we rollout to large customers. Spent the last few days analyzing Health monitor Blogs and Posts. Yes you MS new-style-cloud-monkeys you can do a POST DOC on Exchange Health analyze! But who in the world pays for that? My enterprise customer with their CAL's

    Some stuff i found out even from MS Blogs:

    a) Microsoft decided to take Exchange to Office365. (The Cloud)

    b) Because the OWN people in-house think they are smart they began integrating their Exchange in Management and Monitoring Software complete. At the end they had 1'100 rules. As IT-manager i would say the where too lazy to check the Exchanges each day and played games. (Like the one from Jurassic Park! Like the guys you don't want to manage.)

    c) When first enterprise customer who are NOT stupid and put their data into hand of Americans decided to jump to another solution THEY decided to rollout On premise (Some weird term let’s call it IN-HOUSE) solution and had to cut down the Health Status to 800 for the consumers. The manual of Exchange 2013 printed would have been 14'600 pages! Manual Exchange 100, the rest Health Monitor and how to debug!

    d) I did Clusters, SCR-Replication, DAG, Hardware Load Balancers and

    Exchange 2010 is the dam best version that runs and except SQL one best the best products i have seen. I don't know all but Blogs help me a lot and I do some blogs and share the info we learned.

    There is nothing you have to Monitor and we sometimes approach Exchange 2007/2010 customers who have run 3 years without anything gone corrupt, Defect and nobody did nothing except it did was it has to do. Like a thing you pay money for and except to work.

    2013 and the Self healing function from Photoshop placed in a Groupware

    The concept was some self healing Exchange for 2013. It monitors, does magic things, it even Recycles the hated IIS Application pools and other things like DAg Failover. In

    some cases it stops Services and WORST Constellation and full storage it will Trigger
    a BLUE SCREEN (BOD) on your OS (Yes!). Who knows it will even migrated into the
    CLOUD overnight like W7>W10 so we can all leave work and play golf.<o:p></o:p>

    There has been discussion in IT Pro / All Exchange Blogs / All Mags about the Health Monitors.

    Question: Why do people then always pretend there is no problem and behave like sales with ties. “You are the first one who says that strange never heard."

    Well why don't they stand there and say, yes we coded some crap because we gave it to some academic-super-nerd who will not survive in real world for 7 days...? Sorry customers but move to the Cloud or live with some crap we made for you older guys who don’t like Cloud and maybe have sensitive Data? If I need Cloud I put it to some open source and don't need MS anymore. And please no Exchange on Linux like the SQL. ;-)




    Thursday, March 10, 2016 10:30 PM
  • No not for his case and two customer cases we had. He has done that all. It does not solve it.
    Thursday, March 10, 2016 10:33 PM
  • In AD Users and Computers, go to view and make sure advanced features is selected.  You should then be able to see the folder she describes in the tree.

    Sorry He is talking about deletion of an Exchange Object? And you wan't to explain him how to View "Adanced View" in Active Directory user and Computers? There was a BUG where the SUB OU under the one all talk about was not created (have seen that several times).


    Thursday, March 10, 2016 10:35 PM
  • "There is nothing you have to Monitor and we sometimes approach Exchange 2007/2010 customers who have run 3 years without anything gone corrupt" - agree! It is "amazing" to troubleshoot the healh subsystem (sometimes right after installation) which has been created to relieve administrators from many Exchange-related issues...

    By the way, I wonder was there a real need to create the health service for auto-healing? For example, I've been deploying Exchange since 1999 (Exchange 5.5 ... >) and never thought it lacked such service...

    Regards,

    Michael

    Thursday, March 24, 2016 10:38 AM