none
Microsoft.Exchange.Store.Worker is consuming a lot of CPU Resources - causes 99% CPU load on the server RRS feed

  • Question

  • Hi,

    I have 2 Exchange 2013 CAS/MBX servers, and 2 Exchange 2010 CAS/HUB/MBX servers. Exchange 2010 is not hosting any mailboxes, except for Public Folders that will be migrated soon.

    There are 12 Mailbox databases on Exchange 2013, 6 mounted on each Exchange 2013 server within a DAG. We discovered that 2 months after the beginning of the mailbox migration from 2010 to 2013, the CPU on each Exchange 2013 servers was at 99%, and that some users were starting to complain about issues in Outlook. To solve the issues, we rebooted both servers, which indeed solved the problem and the CPU peak problem... but only for 2 months, since then the CPU is now again at 99% on both servers. The process responsible for this is "Microsoft.Exchange.Store.Worker". There are 6 of them on each server and they typically uses 15-35% of CPU resource (not every Store Worker, it is generally two of them and always the same), which causes the CPU peaks. I know this is the process for each database.

    This is an odd behavior that is not normal I think. We checked the graph on VMware Hosts, and the CPU load is gradually rising every day...

    Both Exchange 2013 Server are at CU22, (we already had this issue when we were at CU15 level).

    Any ideas ?

    Thanks in advance



    Tuesday, July 9, 2019 7:34 AM

All replies

  • Hi,

    I have 2 Exchange 2013 CAS/MBX servers, and 2 Exchange 2010 CAS/HUB/MBX servers. Exchange 2010 is not hosting any mailboxes, except for Public Folders that will be migrated soon.

    There are 12 Mailbox databases on Exchange 2013, 6 mounted on each Exchange 2013 server within a DAG. We discovered that 2 months after the beginning of the mailbox migration from 2010 to 2013, the CPU on each Exchange 2013 servers was at 99%, and that some users were starting to complain about issues in Outlook. To solve the issues, we rebooted both servers, which indeed solved the problem and the CPU peak problem... but only for 2 months, since then the CPU is now again at 99% on both servers. The process responsible for this is "Microsoft.Exchange.Store.Worker". There are 6 of them on each server and they typically uses 15-35% of CPU resource (not every Store Worker, it is generally two of them and always the same), which causes the CPU peaks. I know this is the process for each database.

    This is an odd behavior that is not normal I think. We checked the graph on VMware Hosts, and the CPU load is gradually rising every day...

    Both Exchange 2013 Server are at CU22, (we already had this issue when we were at CU15 level).

    Any ideas ?

    Thanks in advance



    Did you size the servers correctly with the mailbox calculator and follow the virtualization guidance?

    Tuesday, July 9, 2019 11:38 AM
    Moderator
  • Yes, we used the Exchange Calculator and servers are correctly sized.

    There is also no hint in the Event Viewer of any possible issue


    Tuesday, July 9, 2019 12:10 PM
  • Yes, we used the Exchange Calculator and servers are correctly sized.

    There is also no hint in the Event Viewer of any possible issue


    I dont see how many users you have, but maybe its time to scale out? 

    One thing to note. Ensure the VM host bios is  set to high performance.

    Tuesday, July 9, 2019 5:48 PM
    Moderator
  • Hi Mickael,

     

    During the  high CPU workload, does any user mailbox experience slow connection or other performance issue?

     

    In Exchange 2013, single Store Worker Process is only responsible for one database, we can firstly confirm if the specific mailbox database causes the high CPU:

     

    How to determine which store worker process is responsible for which mailbox database?

    Please Note: Since the web site is not hosted by Microsoft, the link may change without notice. Microsoft does not guarantee the accuracy of this information.

     

    It's also suggested use the EXMon tool to verify if high CPU utlization is eat up by specific mailbox.

     

    Microsoft Exchange Server User Monitor For Exchange 2013 and 2016

     

    The antivirus service or any third party scanning software might consume CPU as well, try unloading the AV if it makes any difference.

     

    Regards,

    Kelvin Deng


    Please remember to mark the replies as answers if they helped. If you have feedback for TechNet Subscriber Support, contact tnsf@microsoft.com.


    Click here to learn more. Visit the dedicated forum to share, explore and talk to experts about Microsoft Teams.

    Wednesday, July 10, 2019 3:22 AM
  • Hi ,

     

    I am writing here to confirm with you how the thing going now?

     

    If you need further help, please provide more detailed information, so that we can give more appropriate suggestions

     

    Regards,

    Kelvin Deng


    Please remember to mark the replies as answers if they helped. If you have feedback for TechNet Subscriber Support, contact tnsf@microsoft.com.


    Click here to learn more. Visit the dedicated forum to share, explore and talk to experts about Microsoft Teams.

    Monday, July 15, 2019 8:50 AM
  • Hi Mickael,

     

    During the  high CPU workload, does any user mailbox experience slow connection or other performance issue?

     

    In Exchange 2013, single Store Worker Process is only responsible for one database, we can firstly confirm if the specific mailbox database causes the high CPU:

     

    How to determine which store worker process is responsible for which mailbox database?

    Please Note: Since the web site is not hosted by Microsoft, the link may change without notice. Microsoft does not guarantee the accuracy of this information.

     

    It's also suggested use the EXMon tool to verify if high CPU utlization is eat up by specific mailbox.

     

    Microsoft Exchange Server User Monitor For Exchange 2013 and 2016

     

    The antivirus service or any third party scanning software might consume CPU as well, try unloading the AV if it makes any difference.

     

    Regards,

    Kelvin Deng


    Please remember to mark the replies as answers if they helped. If you have feedback for TechNet Subscriber Support, contact tnsf@microsoft.com.


    Click here to learn more. Visit the dedicated forum to share, explore and talk to experts about Microsoft Teams.

    Hi Kelvin,

    "During the  high CPU workload, does any user mailbox experience slow connection or other performance issue?"

    Yes, at some point when the CPU is at 99%, users are starting to complain about Outlook being slow or freezing.

    There is no Antivirus installed on the Exchange Server (just a TrendMicro ScanMail), and it seems from what I have seen, that it is always the same database that is causing the issue.

    I launched ExMon and also used the Get-StoreUsageStatistics PowerShell command to see that the shared mailboxes are consuming a lot more of CPU Time than the other mailboxes, but I supposed it is "normal" since they are being accessed by several people.

    For your information, we have 600 Mailboxes, and the server is size with 64GB of RAM and 10 Core CPU.

    Kind regards

    Mickael

    Wednesday, July 17, 2019 11:52 AM
  • Hi Mickael,

     

    During the  high CPU workload, does any user mailbox experience slow connection or other performance issue?

     

    In Exchange 2013, single Store Worker Process is only responsible for one database, we can firstly confirm if the specific mailbox database causes the high CPU:

     

    How to determine which store worker process is responsible for which mailbox database?

    Please Note: Since the web site is not hosted by Microsoft, the link may change without notice. Microsoft does not guarantee the accuracy of this information.

     

    It's also suggested use the EXMon tool to verify if high CPU utlization is eat up by specific mailbox.

     

    Microsoft Exchange Server User Monitor For Exchange 2013 and 2016

     

    The antivirus service or any third party scanning software might consume CPU as well, try unloading the AV if it makes any difference.

     

    Regards,

    Kelvin Deng


    Please remember to mark the replies as answers if they helped. If you have feedback for TechNet Subscriber Support, contact tnsf@microsoft.com.


    Click here to learn more. Visit the dedicated forum to share, explore and talk to experts about Microsoft Teams.

    Hi Kelvin,

    "During the  high CPU workload, does any user mailbox experience slow connection or other performance issue?"

    Yes, at some point when the CPU is at 99%, users are starting to complain about Outlook being slow or freezing.

    There is no Antivirus installed on the Exchange Server (just a TrendMicro ScanMail), and it seems from what I have seen, that it is always the same database that is causing the issue.

    I launched ExMon and also used the Get-StoreUsageStatistics PowerShell command to see that the shared mailboxes are consuming a lot more of CPU Time than the other mailboxes, but I supposed it is "normal" since they are being accessed by several people.

    For your information, we have 600 Mailboxes, and the server is size with 64GB of RAM and 10 Core CPU.

    Kind regards

    Mickael

    Are the Outlook clients in cache mode?

    Also, if there one DB that seems to be the issue, consider moving some mailboxes out of it and spread them out a bit. 

    Wednesday, July 17, 2019 11:56 AM
    Moderator