locked
Exchange 2010 MP constantly throws "WebServices connectivity (Internal) transaction failure" RRS feed

 > 

  • Question

  • Question
    Sign in to vote
    0
    Sign in to vote

    I am getting hundreds of alerts daily:

    Alert: WebServices connectivity (Internal) transaction failure - The credentials cannot be used to test Web Services.
    Source: WebServices - <CAS SERVER> (Client Access) - <Site>
    Path: <CAS SERVER FQDN>;<CAS SERVER> (Client Access) - <Site> Last modified by: System Last modified time: 6/2/2010 8:28:57 AM Alert description: The test mailbox was not initialized. Run new-TestCasConnectivityUser.ps1 to ensure that the test mailbox is created.
    Detailed information:

     Local Site:<Site>

    [Microsoft.Exchange.Monitoring.CasHealthCouldNotLogUserException]: The task couldn't sign in with user Domain\extest_d9eb179636ef4. Run Scripts\new-TestCasConnectivityUser.ps1 to verify that the user exists on the Mailbox server <Mailbox Server>

    I have tried these steps:

    1) Ran Get-MailboxServer |  .\new-TestCasConnectivityUser.ps1 which results in: "WARNING: The command completed successfully but no settings of
    '/extest_d9eb179636ef4' have been modified"

    2) Ensured that extest_d9eb179636ef4 is in the "View Only Administrators"

    3) Unlocked the account, and run Test-OwaConnectivity -ClientAccessServer <CAS Server>, which results in a "Sucess"

    A few minutes after unlocking the extest_d9eb179636ef4 account, all may alerts resolve, and then the account gets locked out again, and the cycle restarts.

    What's causing the account to lockout?

    Karl

    http://unlockpowershell.wordpress.com
    -join("6B61726C6D69747363686B65406D742E6E6574"-split"(?<=\G.{2})",19|%{[char][int]"0x$_"})
    Wednesday, June 2, 2010 2:50 PM

Answers

  • Question
    Sign in to vote
    2
    Sign in to vote

    Karl,

    This is a known issue in Exchange 2010. Under certain circumstances customers may notice the monitoring test account (extest_*) is locked out after the connectivity cmdlets haven been running for a short while. The conditions under which this can happen are:
     
    1. Exchange Server 2010 RTM
    2. The ONLY IIS authentication method enabled is Basic
    3. Exchange monitoring is enabled and monitoring cmdlets are being executed at intervals
     
    The root cause of the issue is that multiple cmdlets are executed simultaneously against the same mailbox, and one behavior of these cmdlets is to "ping" the IIS endpoint to discover the supported IIS authentication methods which can trigger up to five AccessDenied errors. This is registered by the Windows Audit System and counted as failed logon attempts. When the count of consecutive AccessDenied reached a pre-determined limit, set through the Windows Group Policy, the account gets locked out.
     
    The issue has been addressed in Exchange 2010 Service Pack 1.

    A possible workaround is to create a Group Policy Object (or Password Settings Object) with a no account lockout policy. Add the dedicated test user account created by the new-TestCasConnectivityUser.ps1 script to this group.

    Depending on the OS version on which you are running your Exchange Server you can find detailed instructions in the technet articles below. I successfully tried the workaround for Windows Server 2008.

    http://technet.microsoft.com/en-us/library/cc773164(WS.10).aspx
    http://technet.microsoft.com/en-us/library/cc770394%28WS.10%29.aspx

    Thanks,

    Henrik

    Henrik N. Jensen [MSFT]
    Friday, August 20, 2010 5:11 PM

All replies

  • Question
    Sign in to vote
    0
    Sign in to vote

    Hi Karl,

    According to the error log, the issue should be caused by this account. Please check if the following tool will help:

    Account Lockout and Management Tools:

    http://www.microsoft.com/downloads/details.aspx?FamilyId=7AF2E69C-91F3-4E63-8629-B999ADDE0B9E&displaylang=en

    Vivian Xing - MSFT
    Thursday, June 3, 2010 6:41 AM
  • Question
    Sign in to vote
    0
    Sign in to vote

    Vivian;

    It might, except for the fact that ythe documentations says not to "use this tool on servers that host network applications or services. Also, you should not use ALockout.dll on Exchange servers, because it may prevent the Exchange store from starting."

    Karl

    http://unlockpowershell.wordpress.com
    -join("6B61726C6D69747363686B65406D742E6E6574"-split"(?<=\G.{2})",19|%{[char][int]"0x$_"})
    Thursday, June 3, 2010 2:02 PM
  • Question
    Sign in to vote
    0
    Sign in to vote
    the account that you need to use to avoid lockout is localsystem.  If you use any other account, you will have this problem.
    Microsoft Corporation
    • Marked as answer by Dan Rogers Wednesday, June 16, 2010 2:59 PM
    • Unmarked as answer by Just Karl Thursday, June 17, 2010 5:04 PM
    Wednesday, June 16, 2010 2:59 PM
  • Question
    Sign in to vote
    0
    Sign in to vote

    Dan;

    What do you mean by this?

    You cannot use "localsystem" - the script "new-TestCasConnectivityUser.ps1" creates a domain account if the account does not already exist.

    I won't unmark your answer just yet, but I don't see HOW it could be an answer....

    I *MAY* have resolved this via http://technet.microsoft.com/en-us/library/ee758052.aspx though

    Karl

    http://unlockpowershell.wordpress.com
    -join("6B61726C6D69747363686B65406D742E6E6574"-split"(?<=\G.{2})",19|%{[char][int]"0x$_"})
    Wednesday, June 16, 2010 3:36 PM
  • Question
    Sign in to vote
    0
    Sign in to vote

    I was wrong - the http://technet.microsoft.com/en-us/library/ee758052.aspx did not fix the alerts.

    Karl

    http://unlockpowershell.wordpress.com
    -join("6B61726C6D69747363686B65406D742E6E6574"-split"(?<=\G.{2})",19|%{[char][int]"0x$_"})
    Wednesday, June 16, 2010 9:31 PM
  • Question
    Sign in to vote
    0
    Sign in to vote

    I have high hopes for this article:

    http://support.microsoft.com/kb/2022687

    Will test this weekend.

    Karl

    http://unlockpowershell.wordpress.com
    -join("6B61726C6D69747363686B65406D742E6E6574"-split"(?<=\G.{2})",19|%{[char][int]"0x$_"})
    Thursday, June 17, 2010 5:04 PM
  • Question
    Sign in to vote
    0
    Sign in to vote

    Our RPC with Certs was already configured as suggested, when I set RPC to ASP.net impersonation, putlook Anywhere broke, so I am still getting hundreds of these alerts daily.

    Karl

    http://unlockpowershell.wordpress.com
    -join("6B61726C6D69747363686B65406D742E6E6574"-split"(?<=\G.{2})",19|%{[char][int]"0x$_"})
    Thursday, June 24, 2010 2:02 PM
  • Question
    Sign in to vote
    2
    Sign in to vote

    Karl,

    This is a known issue in Exchange 2010. Under certain circumstances customers may notice the monitoring test account (extest_*) is locked out after the connectivity cmdlets haven been running for a short while. The conditions under which this can happen are:
     
    1. Exchange Server 2010 RTM
    2. The ONLY IIS authentication method enabled is Basic
    3. Exchange monitoring is enabled and monitoring cmdlets are being executed at intervals
     
    The root cause of the issue is that multiple cmdlets are executed simultaneously against the same mailbox, and one behavior of these cmdlets is to "ping" the IIS endpoint to discover the supported IIS authentication methods which can trigger up to five AccessDenied errors. This is registered by the Windows Audit System and counted as failed logon attempts. When the count of consecutive AccessDenied reached a pre-determined limit, set through the Windows Group Policy, the account gets locked out.
     
    The issue has been addressed in Exchange 2010 Service Pack 1.

    A possible workaround is to create a Group Policy Object (or Password Settings Object) with a no account lockout policy. Add the dedicated test user account created by the new-TestCasConnectivityUser.ps1 script to this group.

    Depending on the OS version on which you are running your Exchange Server you can find detailed instructions in the technet articles below. I successfully tried the workaround for Windows Server 2008.

    http://technet.microsoft.com/en-us/library/cc773164(WS.10).aspx
    http://technet.microsoft.com/en-us/library/cc770394%28WS.10%29.aspx

    Thanks,

    Henrik

    Henrik N. Jensen [MSFT]
    Friday, August 20, 2010 5:11 PM
  • Question
    Sign in to vote
    0
    Sign in to vote

    Let's hope Exchange 2010 SP1 fixes this....

    Karl

    http://unlockpowershell.wordpress.com
    -join("6B61726C6D69747363686B65406D742E6E6574"-split"(?<=\G.{2})",19|%{[char][int]"0x$_"})
    Monday, August 23, 2010 5:17 PM