none
Strange DNS, Group Policy & Active Directory Issues - Can't track down root issue!

    Question

  • For the last few weeks, we've been getting complaints, from our developers, about not being able to authenticate on various systems.  The issues were hit & miss but still problematic enough to warrant our looking into it.  It seems to be getting worse...  I now have new servers that aren't getting group policy updates.  They may get some, like the list of local admins but won't pick up NTFS permissions for folder-access.  Those that pick up the AD group full of local admins have trouble authenticating members of the group.  Some were showing event log entries regarding authentication issues due to being unable to contact an AD DC.  We reloaded that DC but many of the issues still persist.  At this point, I'm running out of places to look for ideas.  I've spent the last week looking up Event Log IDs and looking though their meanings and possible remedies but, again, the issues persist.  It doesn't seem to matter what the OS is.  We've been seeing this on 2008, 2008-R2 & 2012-R2.

    Here are some examples of events I'm seeing.  I can't figure out the root cause(s).

    Log Name:      Application
    Source:        Group Policy Files
    Date:          2/19/2015 2:35:12 PM
    Event ID:      4098
    Task Category: (2)
    Level:         Warning
    Keywords:      Classic
    User:          SYSTEM
    Computer:      H2T8-IOLDP1.HOMENET.local
    Description:
    The computer 'uptime.exe' preference item in the 'APPS (UpTime) {3BF05605-27C0-43AD-AC0F-873B678EB217}' Group Policy Object did not apply because it failed with error code '0x80090006 Invalid Signature.' This error was suppressed.
    Event Xml:
    <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
      <System>
        <Provider Name="Group Policy Files" />
        <EventID Qualifiers="34305">4098</EventID>
        <Level>3</Level>
        <Task>2</Task>
        <Keywords>0x80000000000000</Keywords>
        <TimeCreated SystemTime="2015-02-19T19:35:12.000000000Z" />
        <EventRecordID>1871</EventRecordID>
        <Channel>Application</Channel>
        <Computer>H2T8-IOLDP1.HOMENET.local</Computer>
        <Security UserID="S-1-5-18" />
      </System>
      <EventData>
        <Data>computer</Data>
        <Data>uptime.exe</Data>
        <Data>APPS (UpTime) {3BF05605-27C0-43AD-AC0F-873B678EB217}</Data>
        <Data>0x80090006 Invalid Signature.</Data>
      </EventData>
    </Event>

    Log Name:      Microsoft-Windows-TerminalServices-RemoteConnectionManager/Admin
    Source:        Microsoft-Windows-TerminalServices-RemoteConnectionManager
    Date:          2/19/2015 9:38:13 AM
    Event ID:      20499
    Task Category: None
    Level:         Warning
    Keywords:      
    User:          NETWORK SERVICE
    Computer:      H2T8-IOLDP1.HOMENET.local
    Description:
    Remote Desktop Services has taken too long to load the user configuration from server \\h2s3-addc1.HOMENET.local for user RSickler
    Event Xml:
    <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
      <System>
        <Provider Name="Microsoft-Windows-TerminalServices-RemoteConnectionManager" Guid="{C76BAA63-AE81-421C-B425-340B4B24157F}" />
        <EventID>20499</EventID>
        <Version>0</Version>
        <Level>3</Level>
        <Task>0</Task>
        <Opcode>0</Opcode>
        <Keywords>0x4000000000000000</Keywords>
        <TimeCreated SystemTime="2015-02-19T14:38:13.182363700Z" />
        <EventRecordID>4</EventRecordID>
        <Correlation />
        <Execution ProcessID="1932" ThreadID="2156" />
        <Channel>Microsoft-Windows-TerminalServices-RemoteConnectionManager/Admin</Channel>
        <Computer>H2T8-IOLDP1.HOMENET.local</Computer>
        <Security UserID="S-1-5-20" />
      </System>
      <UserData>
        <EventXML xmlns="Event_NS">
          <ServerName>\\h2s3-addc1.HOMENET.local</ServerName>
          <UserName>RSickler</UserName>
        </EventXML>
      </UserData>
    </Event>

    Note that these servers are sitting in OUs that are full of other servers that don't have these issues.  These GPOs have been in place for years.  I suspect there's a deeper issue with AD, GP or a combination thereof.  The group policy issues seem to only affect freshly loaded servers...
    Friday, February 20, 2015 12:37 PM

Answers

  • Late Friday, a colleague and I figured out the root cause for one of the GPO failures I listed from Event Viewer in my original post.  The Event ID 1871 (Invalid Signature) is due to the "Secure Negotiate" feature in Server 2012/2012-R2.  We have an older EMC NAS which give us issues now and then.  This time, it was due to this updated feature in SMB 3.  So, that issue has been fixed by disabling Secure Negotiate on Server 2012-R2 servers.  The fix is listed below for that...

    Set-ItemProperty -Path "HKLM:\SYSTEM\CurrentControlSet\Services\LanmanWorkstation\Parameters" RequireSecureNegotiate -Value 0 -Force

    I'm still seeing issues so I'll post the data Meinolf requested in my next post.

    Monday, February 23, 2015 1:34 PM
  • Hello,

    please check for the error message shown on multiple DCs:

    [1] Problem: Missing Expected Value

    •              Base Object: CN=DC2,OU=Domain Controllers,DC=HOMENET,DC=local
    •              Base Object Description: "DC Account Object"
    •              Value Object Attribute Name: msDFSR-ComputerReferenceBL
    •              Value Object Description: "SYSVOL FRS Member Object"
    •              Recommended Action: See Knowledge Base Article: Q312862
    • with the following article http://technet.microsoft.com/en-us/library/cc794759(WS.10).aspx

      For "The event logging service encountered an error (res=3) while opening log file for channel System. Trying again using default log file path %SystemRoot%\System32\Winevt\Logs\System.evtx." please compare with the article https://social.msdn.microsoft.com/Forums/en-US/2cac1064-f970-4c47-b0ea-296d53b23d1d/windows-event-logs-not-getting-logged-to-a-custom-path?forum=quebeccomponentsforum if you have redirected something.

      As FORWARDERS use ONLY external DNS servers, not internal ones or do you have a specific need for this, multiple DNS Servers this is an example only:

      TEST: Forwarders/Root hints (Forw)
    •                   Recursion is enabled
    •                   Forwarders Information:
    •                      10.224.210.7 (DC1) [Valid]
    •                      204.11.138.11 (<name unavailable>) [Valid]
    •                      204.11.138.12 (<name unavailable>) [Invalid (unreachable)]
    •                      8.8.4.4 (<name unavailable>) [Valid]
    •                      8.8.8.8 (<name unavailable>) [Valid]


    Best regards

    Meinolf Weber

    MVP, MCP, MCTS

    Microsoft MVP - Directory Services

    My Blog: http://blogs.msmvps.com/MWeber

    Disclaimer: This posting is provided AS IS with no warranties or guarantees and confers no rights.

    Twitter:  

    Monday, February 23, 2015 2:48 PM

All replies

  • Hi Rob,

    First thing, please provide run "dcdiag /e" on domain controller and post the result here. Run the below commands from your client machine.

    nltest /sc_query:<domain name>

    nltest /dclist:<domain name>

    Thanks,

    Umesh.S.K

    Friday, February 20, 2015 12:42 PM
  • Hi Rob,

    First thing, please provide run "dcdiag /e" on domain controller and post the result here. Run the below commands from your client machine.

    nltest /sc_query:<domain name>

    nltest /dclist:<domain name>

    Thanks,

    Umesh.S.K

    Didn't know which DC you wanted me to run the DIAG on but I grabbed a random one and did it.  This forum nags about the size of the post so I had to place it into PasteBin: H2S3-ADDC2_dcdiag.txt

    H2T8-IOLRPC1_dclist.txt:

    Get list of DCs in domain 'homenet.local' from '\\H2S2-ADDC2.HOMENET.local'.
               DC2.HOMENET.local        [DS] Site: WestChester
        H2S2-ADDC2.HOMENET.local        [DS] Site: AtlantaStaging
               dc1.HOMENET.local [PDC]  [DS] Site: WestChester
         H2P-ADDC1.HOMENET.local        [DS] Site: AtlantaH2P
         H2P-ADDC2.HOMENET.local        [DS] Site: AtlantaH2P
         H3R-ADDC1.HOMENET.local        [DS] Site: AtlantaH3R
         H3R-ADDC2.HOMENET.local        [DS] Site: AtlantaH3R
        H2S3-ADDC1.HOMENET.local        [DS] Site: AtlantaStaging
        H2S3-ADDC2.HOMENET.local        [DS] Site: AtlantaStaging
        H2S2-ADDC1.HOMENET.local        [DS] Site: AtlantaStaging
    The command completed successfully
    

    H2T8-IOLRPC1_sc_query.txt:

    Flags: 30 HAS_IP  HAS_TIMESERV 
    Trusted DC Name \\h2s3-addc1.HOMENET.local 
    Trusted DC Connection Status Status = 0 0x0 NERR_Success
    The command completed successfully
    

    Friday, February 20, 2015 1:07 PM
  • Hello,

    assure that no firewall is blocking connection for AD required ports as listed in https://technet.microsoft.com/en-us/library/dd772723(WS.10).aspx

    ----------------------------

    You have error about not connect setup from AD sites and services with the used subnets in your network and linking them to the correct site, please check this in AD sites and services and also have the DCs placed correct to the site they belong to.

    "During the past 4.20 hours there have been 83 connections to this Domain Controller from client machines whose IP addresses don't map to any of the existing sites in the enterprise. Those clients, therefore, have undefined sites and may connect to any Domain Controller including those that are in far distant locations from the clients. A client's site is determined by the mapping of its subnet to one of the existing sites. To move the above clients to one of the sites, please consider creating subnet object(s) covering the above IP addresses with mapping to one of the existing sites.  The names and IP addresses of the clients in question have been logged on this computer in the following log file '%SystemRoot%\debug\netlogon.log' and, potentially, in the log file '%SystemRoot%\debug\netlogon.bak' created if the former log becomes full. The log(s) may contain additional unrelated debugging information. To filter out the needed information, please search for lines which contain text 'NO_CLIENT_SITE:'. The first word after this string is the client name and the second word is the client IP address. The maximum size of the log(s) is controlled by the following registry DWORD value 'HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Netlogon\Parameters\LogFileMaxSize'; the default is 20000000 bytes.  The current maximum size is 20000000 bytes.  To set a different maximum size, create the above registry value and set the desired maximum size in bytes."

    ----------------------------

    This error is about a not run adprep /rodcprep:

    Starting test: NCSecDesc
    •          Error NT AUTHORITY\ENTERPRISE DOMAIN CONTROLLERS doesn't have
    •             Replicating Directory Changes In Filtered Set
    •          access rights for the naming context:
    •          DC=ForestDnsZones,DC=HOMENET,DC=local
    •          Error NT AUTHORITY\ENTERPRISE DOMAIN CONTROLLERS doesn't have
    •             Replicating Directory Changes In Filtered Set
    •          access rights for the naming context:

      ----------------------------

      So either run the command on a DC or ignore this error.

      Please provide also the following data as file:

      ipconfig /all >c:\ipconfig.log [all DCs]
      dcdiag /v /c /d /e /s:dcname >c:\dcdiag.log
      repadmin /showrepl dc* /verbose /all /intersite >c:\repl.log  ["dc* is a place holder for the starting name of the DCs if they all begin the same (if more then one DC exists)]
      dnslint /ad /s "DCipaddress" (http://support.microsoft.com/kb/321045)
      ADREPLSTATUS: http://www.microsoft.com/en-us/download/details.aspx?id=30005 can also be exported to file.

      As the output will become large, DON'T post them into the thread, please use Windows Sky Drive(with open access!) https://skydrive.live.com and add the link from it here. Also the /e in dcdiag scans the complete forest, so better run it on COB.


    Best regards

    Meinolf Weber

    MVP, MCP, MCTS

    Microsoft MVP - Directory Services

    My Blog: http://blogs.msmvps.com/MWeber

    Disclaimer: This posting is provided AS IS with no warranties or guarantees and confers no rights.

    Twitter:  

    Monday, February 23, 2015 8:25 AM
  • Late Friday, a colleague and I figured out the root cause for one of the GPO failures I listed from Event Viewer in my original post.  The Event ID 1871 (Invalid Signature) is due to the "Secure Negotiate" feature in Server 2012/2012-R2.  We have an older EMC NAS which give us issues now and then.  This time, it was due to this updated feature in SMB 3.  So, that issue has been fixed by disabling Secure Negotiate on Server 2012-R2 servers.  The fix is listed below for that...

    Set-ItemProperty -Path "HKLM:\SYSTEM\CurrentControlSet\Services\LanmanWorkstation\Parameters" RequireSecureNegotiate -Value 0 -Force

    I'm still seeing issues so I'll post the data Meinolf requested in my next post.

    Monday, February 23, 2015 1:34 PM
  • Hello,

    assure that no firewall is blocking connection for AD required ports as listed in https://technet.microsoft.com/en-us/library/dd772723(WS.10).aspx

    ----------------------------

    You have error about not connect setup from AD sites and services with the used subnets in your network and linking them to the correct site, please check this in AD sites and services and also have the DCs placed correct to the site they belong to.

    "During the past 4.20 hours there have been 83 connections to this Domain Controller from client machines whose IP addresses don't map to any of the existing sites in the enterprise. Those clients, therefore, have undefined sites and may connect to any Domain Controller including those that are in far distant locations from the clients. A client's site is determined by the mapping of its subnet to one of the existing sites. To move the above clients to one of the sites, please consider creating subnet object(s) covering the above IP addresses with mapping to one of the existing sites.  The names and IP addresses of the clients in question have been logged on this computer in the following log file '%SystemRoot%\debug\netlogon.log' and, potentially, in the log file '%SystemRoot%\debug\netlogon.bak' created if the former log becomes full. The log(s) may contain additional unrelated debugging information. To filter out the needed information, please search for lines which contain text 'NO_CLIENT_SITE:'. The first word after this string is the client name and the second word is the client IP address. The maximum size of the log(s) is controlled by the following registry DWORD value 'HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Netlogon\Parameters\LogFileMaxSize'; the default is 20000000 bytes.  The current maximum size is 20000000 bytes.  To set a different maximum size, create the above registry value and set the desired maximum size in bytes."

    ----------------------------

    This error is about a not run adprep /rodcprep:

    Starting test: NCSecDesc
    •          Error NT AUTHORITY\ENTERPRISE DOMAIN CONTROLLERS doesn't have
    •             Replicating Directory Changes In Filtered Set
    •          access rights for the naming context:
    •          DC=ForestDnsZones,DC=HOMENET,DC=local
    •          Error NT AUTHORITY\ENTERPRISE DOMAIN CONTROLLERS doesn't have
    •             Replicating Directory Changes In Filtered Set
    •          access rights for the naming context:

      ----------------------------

      So either run the command on a DC or ignore this error.

      Please provide also the following data as file:

      ipconfig /all >c:\ipconfig.log [all DCs]
      dcdiag /v /c /d /e /s:dcname >c:\dcdiag.log
      repadmin /showrepl dc* /verbose /all /intersite >c:\repl.log  ["dc* is a place holder for the starting name of the DCs if they all begin the same (if more then one DC exists)]
      dnslint /ad /s "DCipaddress" (http://support.microsoft.com/kb/321045)
      ADREPLSTATUS: http://www.microsoft.com/en-us/download/details.aspx?id=30005 can also be exported to file.

      As the output will become large, DON'T post them into the thread, please use Windows Sky Drive(with open access!) https://skydrive.live.com and add the link from it here. Also the /e in dcdiag scans the complete forest, so better run it on COB.


    Best regards

    Meinolf Weber

    MVP, MCP, MCTS

    Microsoft MVP - Directory Services

    My Blog: http://blogs.msmvps.com/MWeber

    Disclaimer: This posting is provided AS IS with no warranties or guarantees and confers no rights.

    Twitter:  

    Info you requested:

    ipconfig_dcs.txt

    dcdiag.txt

    repl.log

    dnslint.htm

    ADREPLSTATUS: ADReplicationStatus.2015.2.23.9.21.16.csv ADReplicationStatusToolData.zip

    Monday, February 23, 2015 2:31 PM
  • From what I can see, most of the GPOs are getting processed.  Since we fixed the one issue on Firday, I'm only seeing one GPO that's not applying.  There are no errors logged in Event Viewer and I don't see any errors during a GPUPDATE /FORCE.  Moreover, GPRESULT /R shows nothing out of the ordinary.  This particular GPO is pretty simple.  It has worked for many years and has stopped working now.  All it does is set up NTFS permissions on 2 folders.  I'm not sure if it's the GPO itself or something bigger.  Hopefully that data I just posted a bit ago will help determine the issue.
    Monday, February 23, 2015 2:37 PM
  • Hello,

    please check for the error message shown on multiple DCs:

    [1] Problem: Missing Expected Value

    •              Base Object: CN=DC2,OU=Domain Controllers,DC=HOMENET,DC=local
    •              Base Object Description: "DC Account Object"
    •              Value Object Attribute Name: msDFSR-ComputerReferenceBL
    •              Value Object Description: "SYSVOL FRS Member Object"
    •              Recommended Action: See Knowledge Base Article: Q312862
    • with the following article http://technet.microsoft.com/en-us/library/cc794759(WS.10).aspx

      For "The event logging service encountered an error (res=3) while opening log file for channel System. Trying again using default log file path %SystemRoot%\System32\Winevt\Logs\System.evtx." please compare with the article https://social.msdn.microsoft.com/Forums/en-US/2cac1064-f970-4c47-b0ea-296d53b23d1d/windows-event-logs-not-getting-logged-to-a-custom-path?forum=quebeccomponentsforum if you have redirected something.

      As FORWARDERS use ONLY external DNS servers, not internal ones or do you have a specific need for this, multiple DNS Servers this is an example only:

      TEST: Forwarders/Root hints (Forw)
    •                   Recursion is enabled
    •                   Forwarders Information:
    •                      10.224.210.7 (DC1) [Valid]
    •                      204.11.138.11 (<name unavailable>) [Valid]
    •                      204.11.138.12 (<name unavailable>) [Invalid (unreachable)]
    •                      8.8.4.4 (<name unavailable>) [Valid]
    •                      8.8.8.8 (<name unavailable>) [Valid]


    Best regards

    Meinolf Weber

    MVP, MCP, MCTS

    Microsoft MVP - Directory Services

    My Blog: http://blogs.msmvps.com/MWeber

    Disclaimer: This posting is provided AS IS with no warranties or guarantees and confers no rights.

    Twitter:  

    Monday, February 23, 2015 2:48 PM
  • Hello,

    please check for the error message shown on multiple DCs:

    [1] Problem: Missing Expected Value

    •              Base Object: CN=DC2,OU=Domain Controllers,DC=HOMENET,DC=local
    •              Base Object Description: "DC Account Object"
    •              Value Object Attribute Name: msDFSR-ComputerReferenceBL
    •              Value Object Description: "SYSVOL FRS Member Object"
    •              Recommended Action: See Knowledge Base Article: Q312862
    • with the following article http://technet.microsoft.com/en-us/library/cc794759(WS.10).aspx

      For "The event logging service encountered an error (res=3) while opening log file for channel System. Trying again using default log file path %SystemRoot%\System32\Winevt\Logs\System.evtx." please compare with the article https://social.msdn.microsoft.com/Forums/en-US/2cac1064-f970-4c47-b0ea-296d53b23d1d/windows-event-logs-not-getting-logged-to-a-custom-path?forum=quebeccomponentsforum if you have redirected something.

      As FORWARDERS use ONLY external DNS servers, not internal ones or do you have a specific need for this, multiple DNS Servers this is an example only:

      TEST: Forwarders/Root hints (Forw)
    •                   Recursion is enabled
    •                   Forwarders Information:
    •                      10.224.210.7 (DC1) [Valid]
    •                      204.11.138.11 (<name unavailable>) [Valid]
    •                      204.11.138.12 (<name unavailable>) [Invalid (unreachable)]
    •                      8.8.4.4 (<name unavailable>) [Valid]
    •                      8.8.8.8 (<name unavailable>) [Valid]


    Best regards

    Meinolf Weber

    MVP, MCP, MCTS

    Microsoft MVP - Directory Services

    My Blog: http://blogs.msmvps.com/MWeber

    Disclaimer: This posting is provided AS IS with no warranties or guarantees and confers no rights.

    Twitter:  

    I changed the event logs to point to their default locations.  Not sure why they were pointing to a subfolder.  Could have been something set up for testing....  Anyway, I found the GPO and disabled that part of it.

    Not entirely sure what you're talking about with the DNS servers.  In which report did you see that info?  What should they be?  Why would they ever have public DNS servers listed?



    Monday, February 23, 2015 5:03 PM
  • Hello,

    the FORWARDERS entry is in the dcdiag Output at the end. There are all DNS Settings listed. Please check the forwarders in the DNS Management console. They are set manually.


    Best regards

    Meinolf Weber

    MVP, MCP, MCTS

    Microsoft MVP - Directory Services

    My Blog: http://blogs.msmvps.com/MWeber

    Disclaimer: This posting is provided AS IS with no warranties or guarantees and confers no rights.

    Twitter:  

    Monday, February 23, 2015 8:02 PM