locked
Issue with cross-forest trust after been working for 6+ months RRS feed

  • Question

  • Hi all

    Hoping to get some help on an issue that started today and so far has got me scratching my head. 

    We started getting calls from our user base that they could not access mapped drives and desktop shortcuts that points to file servers residing on a different AD forest. They get presented with a login prompt which never happened before.

    We can ping domainB.local for the most part, they have some domain controllers around the world we cant connect to.

    nslookup domainB.local also works fine. we get about 8-10 domain controllers where we can only get to 6 of them (UK and Germany)

    Other troubleshooting info:
    using \\SERVERIP\share seems to work. users do not get asked for creds
    Opening AD Users and computers on our main Domain Controller we cant change domain to domainB.local we get something about username and password. doing this from the other side seem to work.

    hope someone has heard of this before and can offer some assistance.

    myself and the team i work for suspect something in DNS but then again we can ping and resolve just fine. users from DomainB.local do not have issues accessing resources our on our side.

    Monday, November 10, 2014 3:51 PM

Answers

  • Hi Ace,

    We have some 2003 domain controllers on our network still but not the PDC DC. the other side has no 2003 domain controllers but they do have a RODC in their DR site that we cant get to.

    The trust is not removed yet as we are note sure what actually broke the damn thing :) We did remove a trust but it was a separate thing, it was just along the same times that the trust broke.

    Would there be any chance we could do some offline discussion about this? Skype, email etc?

    Regards

    I wish I could. I'm just so jammed up at this time working on a contract for a large university Office 365 rollout. I'm so backed up, that I even have a paid customer that's wondering why I haven't worked on something last night that I was supposed to work on from Monday yet. And it's Thanksgiving. I don't know what country you are in, but if you are in the US, I think you can understand that I am the official T-day cook. :-)

    Honestly, my best, inexpensive suggestion at this point is to call Microsoft support for them to sort it out. It's only about $275 + tax. I know you must get this done. I think this is reasonable option. What do you think?


    Ace Fekay
    MVP, MCT, MCSE 2012, MCITP EA & MCTS Windows 2008/R2, Exchange 2013, 2010 EA & 2007, MCSE & MCSA 2003/2000, MCSA Messaging 2003
    Microsoft Certified Trainer
    Microsoft MVP - Directory Services
    Complete List of Technical Blogs: http://www.delawarecountycomputerconsulting.com/technicalblogs.php

    This posting is provided AS-IS with no warranties or guarantees and confers no rights.

    FaceBook Twitter LinkedIn

    Friday, November 28, 2014 4:56 AM

All replies

  • Hi,

    Were there any firewall/network changes on the path between DCs on the 2 forests?

    Can you still validate the trust on both sides?

    Regards,

    Calin

    Monday, November 10, 2014 9:39 PM
  • Hi Calin

    not as far as i am aware, everyone says they did not change anything. We can validate the trust on both sides no problems.

    We did loose MPLS connectivity for 24 hours or so between theur office and the rest of our offices. we are thinking if the shared secret is out of sync

    Tuesday, November 11, 2014 10:16 PM
  • G'day Ronnie,

    Using the IP address to access resources on Windows kit forces the authentication mechanism to use NTLM rather than Kerberos.

    So the above problem is 100% a Kerberos failure which fails back to NTLM giving your users a prompt. This could be caused by several things.

    Best way to get to the bottom of this is via a network capture. Install Network Monitor on a client with the issue and kick off a capture and choose/apply the standard filter "Authentication Traffic"

    The capture might look a little scary a first but dig through the traffic and you should get an idea as to what Kerberos is doing for example; talking to the wrong DC, unable to find the realm (name resolution) etc.

    Let me know how you go and have fun with the three headed dog (a.k.a Kerberos)

    Good luck!

    Matt Cockerill - Adelon Consulting

    Tuesday, November 11, 2014 10:55 PM
  • Hi Matt

    So finally i have some spare time to reply back. We have opened a support case with MS. Really weird that users from the CLGROUP.LOCAL domain can access our network shares but the users from AD.DNSARROW.CO.UK cant access theirs, we get prompted for credentials.

    BUT we have found a workaround. at the creds prompt if you use UPN login and not Username2000 then it works.

    Go figure :) - I will get a packet capture going tomorrow and have a deeper look.

    The domain AD.DNSARROW.CO.UK created another EXTERNAL trust on Thursday, although no support calls until following Monday. could this have played a role?

    Thursday, November 13, 2014 11:36 PM
  • If UPN works and not NTLM (or also known as NetBIOS method using the domain\user method), then it means that NetBIOS support failed.

    Do you have WINS running to support NetBIOS traffic and functionality? Note that this will also cause issues with third party apps that use NetBIOS/NTLM and not Kerb auth.

    And note, that External trusts are soley NetBIOS (NTLM) based. Forest trusts are Kerb based. Is that what you had, a Forest trust? Then that would indicate what you are seeing, especially if you and the trusted partner did not support NetBIOS resolution between your entities.


    Ace Fekay
    MVP, MCT, MCSE 2012, MCITP EA & MCTS Windows 2008/R2, Exchange 2013, 2010 EA & 2007, MCSE & MCSA 2003/2000, MCSA Messaging 2003
    Microsoft Certified Trainer
    Microsoft MVP - Directory Services
    Complete List of Technical Blogs: http://www.delawarecountycomputerconsulting.com/technicalblogs.php

    This posting is provided AS-IS with no warranties or guarantees and confers no rights.

    FaceBook Twitter LinkedIn

    Friday, November 14, 2014 4:55 AM
  • Hi Ace

    2 days ago we removed a trust to a new partner but we left the one in place between AD.DNSARROW.CO.UK and CLGROUP.LOCAL

    Just tried logging in to my Win7 machine using UPN format and opening up a network share on the CLGROUP.LOCAL domain. still does not work but typing in the UPN login details again then works.


    The trust was verified both sides using the GUI. from AD.DNSARROW.CO.UK using the command line gives a funny error

    C:\Users\USER>netdom trust ad.dnsarrow.co.uk /d:clgroup.local /verify
    The command failed to complete successfully.

    Friday, November 14, 2014 9:04 AM
  • Hi Ace

    2 days ago we removed a trust to a new partner but we left the one in place between AD.DNSARROW.CO.UK and CLGROUP.LOCAL

    Just tried logging in to my Win7 machine using UPN format and opening up a network share on the CLGROUP.LOCAL domain. still does not work

    but typing in the UPN login details again then works.

    I assume you mean NTLM format, also known as the Pre-Windows 2000 method, doesn't work, such as using the format:
    NetBIOSdomainName\username

    If that doesn't work, once again, were you or are you supporting NetBIOS names across subnets (routers), such as using WINS? Otherwise, I can see why the pre-Windows 2000 method is not working, unless you have a DC from the trusted domain on your own subnet.

    NetBIOS support across subnets also gives you the ability to view all machines in Network Neighborhood on other subnets, among other things.



    Ace Fekay
    MVP, MCT, MCSE 2012, MCITP EA & MCTS Windows 2008/R2, Exchange 2013, 2010 EA & 2007, MCSE & MCSA 2003/2000, MCSA Messaging 2003
    Microsoft Certified Trainer
    Microsoft MVP - Directory Services
    Complete List of Technical Blogs: http://www.delawarecountycomputerconsulting.com/technicalblogs.php

    This posting is provided AS-IS with no warranties or guarantees and confers no rights.

    FaceBook Twitter LinkedIn

    Saturday, November 15, 2014 4:19 AM
  • Hi Ace

    We used to have WINS on our side ad.dnsarrow.co.uk but about a year ago we removed it. We introduced the trust with clgroup.local about 6 months ago now.

    We know that we have a MPLS network blip for a few hours a few weeks ago that cut us off from the clgroup.local domain. not sure if this could have screwed up the shared password?

    I am trying to tell my management to reset or delete and create the trust again. they are worried it wont come back up again. should they be?

    Friday, November 21, 2014 12:00 PM
  • Hi ace

    Wireshark capture from my own laptop trying to access a fileshare on a server on the other side of the trust shows this error, have you seen that before? my google search did not lead to a solution :(


    Friday, November 21, 2014 3:00 PM
  • I'm not sure if it messed up the secured password or not, but this is new. I haven't seen this error yet. Looking it up, it appears that the "integrity check" may be due to a mixed Windows 2003 and 2008 or newer scenario and the Windows 2003 KDC can't decrypt the kerb ticket that was issued by a Windows 2008 DC.

    KRB_AP_ERR_BAD_INTEGRITY error when server tries to delegate in mixed Read-Only DC and Windows Server 2003 DC environment http://support.microsoft.com/kb/2360265

    So I wonder if removing that trust did it because it's finding another path to the trusted domain (unlikely), or something with the DCs changed. Is there an RODC in the mix, too?

    Late edit: Apparently the resolution is to upgrade all of the DCs to at least all Windows 2008 DCs.


    Ace Fekay
    MVP, MCT, MCSE 2012, MCITP EA & MCTS Windows 2008/R2, Exchange 2013, 2010 EA & 2007, MCSE & MCSA 2003/2000, MCSA Messaging 2003
    Microsoft Certified Trainer
    Microsoft MVP - Directory Services
    Complete List of Technical Blogs: http://www.delawarecountycomputerconsulting.com/technicalblogs.php

    This posting is provided AS-IS with no warranties or guarantees and confers no rights.

    FaceBookTwitterLinkedIn


    Sunday, November 23, 2014 2:45 AM
  • Hi Ace,

    We have some 2003 domain controllers on our network still but not the PDC DC. the other side has no 2003 domain controllers but they do have a RODC in their DR site that we cant get to.

    The trust is not removed yet as we are note sure what actually broke the damn thing :) We did remove a trust but it was a separate thing, it was just along the same times that the trust broke.

    Would there be any chance we could do some offline discussion about this? Skype, email etc?

    Regards

    Tuesday, November 25, 2014 1:55 PM
  • Hi Ace,

    We have some 2003 domain controllers on our network still but not the PDC DC. the other side has no 2003 domain controllers but they do have a RODC in their DR site that we cant get to.

    The trust is not removed yet as we are note sure what actually broke the damn thing :) We did remove a trust but it was a separate thing, it was just along the same times that the trust broke.

    Would there be any chance we could do some offline discussion about this? Skype, email etc?

    Regards

    I wish I could. I'm just so jammed up at this time working on a contract for a large university Office 365 rollout. I'm so backed up, that I even have a paid customer that's wondering why I haven't worked on something last night that I was supposed to work on from Monday yet. And it's Thanksgiving. I don't know what country you are in, but if you are in the US, I think you can understand that I am the official T-day cook. :-)

    Honestly, my best, inexpensive suggestion at this point is to call Microsoft support for them to sort it out. It's only about $275 + tax. I know you must get this done. I think this is reasonable option. What do you think?


    Ace Fekay
    MVP, MCT, MCSE 2012, MCITP EA & MCTS Windows 2008/R2, Exchange 2013, 2010 EA & 2007, MCSE & MCSA 2003/2000, MCSA Messaging 2003
    Microsoft Certified Trainer
    Microsoft MVP - Directory Services
    Complete List of Technical Blogs: http://www.delawarecountycomputerconsulting.com/technicalblogs.php

    This posting is provided AS-IS with no warranties or guarantees and confers no rights.

    FaceBook Twitter LinkedIn

    Friday, November 28, 2014 4:56 AM