none
DNS lookups timing out for .info domains only

    Question

  • We're seeing a weird issue with our DNS resolvers where DNS lookups for .info domains are timing out, but only .info domains. All other GTLD and ccTLD lookups work fine so I doubt it's a fault with the DNS server itself, but I'm wondering if anyone else is seeing the same issue.

    In our setup we have a couple of 2012 servers setup to purely run as resolvers (in different physical locations), and which hold no DNS zones (we have separate authoritative servers). Lookups are performed recursively via root hints rather than using any forwarders.

    We have monitoring setup which connects to the resolvers and initiates a set of lookup requests to confirm the response matches what we would expect to see, and most of the time everything works fine. Now and again however we'll find that one of the resolvers (it can be either of them, but doesn't tend to be both at the same time) will start failing their montoring tests for .info domains, but not any other domains. If I manually do an nslookup to that server I get the same result, all queries for .info domains time out, all queries for anything else return correct results.

    It doesn't matter where the .info is stored either. For instance from our monitoring I check :

    mydomain.info (held on our authoritative DNS Server)
    myotherdomain.info (held on a 3rd parties DNS Server)
    mydomain.net (held on our authoritative DNS Server)

    and both .info's fail, but the .net contines to work fine.

    It will continue to fail for hours (current time for latest incident is 15 hours), and then eventually recover, but only happens periodically. I've probably seen it happen on and off for months, perhaps even years (on a previous build of the server obviously) but never gotten to the bottom of what's causing it since the infrequency makes diagnosis tricky.

    Friday, July 26, 2013 9:37 AM

Answers

All replies

  • Ketith,

    What forwarders are you using? Have you tried changing them to something else, such as 4.2.2.3 & 4.2.2.4? Or if not  using forwarders, give it a shot.

    And when running nslookup testing it, run nslookup in interactive mode by just typing in nslookup, then enter, then try changing servers to see if the results are the same, such as the ones above. Example:

    c:\>nslookup
    >something.info
    (results)
    >server 4.2.2.4
    (hit arrow up twice to re-run something.info)
    >server 8.8.8.8
    (hit arrow up twice to re-run something.info)

    Same results?


    Ace Fekay
    MVP, MCT, MCITP/EA, MCTS Windows 2008/R2 & Exchange 2007, Exchange 2010 EA, MCSE & MCSA 2003/2000, MCSA Messaging 2003
    Microsoft Certified Trainer
    Microsoft MVP - Directory Services
    Technical Blogs & Videos: http://www.delawarecountycomputerconsulting.com/

    This post is provided AS-IS with no warranties or guarantees and confers no rights.

    FaceBook Twitter LinkedIn

    Saturday, July 27, 2013 3:54 AM
  • We're not using forwarders on those servers, just root hint lookups. In effect these servers are setup to BE resolvers, and our other DNS servers use them for their resolvers. The idea being that we maintain control and don't have to rely on a 3rd parties DNS setup, while ensuring we don't have all of our servers doing recursive lookups.

    I can't remember whether I've tried querying an external DNS server in the past, but certainly if I do an nslookup to our other DNS resolver that works correctly. It tends to be just one of them at a time that has the issue, and when it happens it's only when resolving .info domains. When it happens again I'll try some external resolvers, but I've no idea how long it'll be before that happens, probably weeks if not months.

    I don't know how DNS Server chooses which root server to query, and whether it picks one, uses it exclusively for a while and then picks again, or picks one each and every time. If the former is the case so it's quite possible to not hit a particular root hint server for a while, it's almost as if one of the root hint destinations doesn't correctly contain the .info root data, so fails to return the correct results, but you'd think other people would have the same issue if that was the case and there'd be some mention from others online.

    Saturday, July 27, 2013 6:58 AM
  • I'm curious if it happens with an external resolver, too.

    As for Roots, that's an interative process, not recursive such as when you directly query a specific DNS. Recursive says, "I want the answer and nothing but the answer. Whereas an interative query says, "If you don't have the answer, give me a referral to another that does." Therefore, usiong the Roots is an interative process and are hit one at a time until an answer comes up, so to speak, starting with a query for who's authoritative for the TLD, such as "com" then works it's way backwards. So it's not a matter of what to choose, rather what gets referred. It's beyond your control. It's how DNS works. Here's more on the process and differences.

    DNS Recursive Queries vs Iterative Queries
    Published by Ace Fekay, MCT, MVP DS on Nov 12, 2009 at 6:55 PM  1155  0  
    http://msmvps.com/blogs/acefekay/archive/2009/11/12/dns-recursive-queries-vs-iterative-queries.aspx


    Ace Fekay
    MVP, MCT, MCITP/EA, MCTS Windows 2008/R2 & Exchange 2007, Exchange 2010 EA, MCSE & MCSA 2003/2000, MCSA Messaging 2003
    Microsoft Certified Trainer
    Microsoft MVP - Directory Services
    Technical Blogs & Videos: http://www.delawarecountycomputerconsulting.com/

    This post is provided AS-IS with no warranties or guarantees and confers no rights.

    FaceBook Twitter LinkedIn

    Sunday, July 28, 2013 8:42 PM
  • Difficult to test since it happens so infrequently, not getting an error for a month or two wouldn't necessarily indicate there wasn't an issue. Will see if one of our AD servers is using a third party forwarder and setup monitoring on that as well to compare.

    Yeah I'm aware of the difference between the two. What I meant was that the initial query to the root DNS servers uses the root hints, from where it is then told which servers handle .info. If that query gives an invalid response then my DNS server obviously wouldn't be able to query the .info servers. Of course it could also be something at the second interaction when I query the Afilias servers.

    Thinking about it I wonder if I'm hitting a query limit on Afilias's servers, eg x queries in y hours, or more accurately normally coming close to hitting it and occasionally hitting the limit when more users than normal query .info domains. I know different registries have different limits, so that's probably more plausible than the root of Afilias servers having issues and no one else having the same problem. I'll have to check the frequency settings on our monitoring system and what the cache/refresh settings are currently set to in DNS.

    Sunday, July 28, 2013 9:22 PM
  • I misunderstood your Roots question. Sorry.

    As for Afilias, apparently they doi have limits. Theres is a PDF mentuinung that. I founf a Twitter post also referencing that. Posting from my phone,  it's difficult to provide the direct links so here is the Google query results where I saw them - I hope the link shows up correctly in my post.

    https://www.google.com/search?client=ms-android-verizon&source=android-browser-type&hl=en-US&ei=bZX1Uc67L9ep4AOquoCwAQ&q=afilias+query+limits+to+.info&oq=afilias+query+limits+to+.info&gs_l=mobile-gws-serp.3...52312.64098.0.64910.26.24.2.0.0.0.268.2082.17j5j1.23.0....0...1c.1.22.mobile-gws-serp..22.4.540.zo7NiLsXDHk&rlz=1Y1UXZM_enUS514US514


    Ace Fekay
    MVP, MCT, MCITP/EA, MCTS Windows 2008/R2 & Exchange 2007, Exchange 2010 EA, MCSE & MCSA 2003/2000, MCSA Messaging 2003
    Microsoft Certified Trainer
    Microsoft MVP - Directory Services
    Technical Blogs & Videos: http://www.delawarecountycomputerconsulting.com/

    This post is provided AS-IS with no warranties or guarantees and confers no rights.

    FaceBook Twitter LinkedIn

    Sunday, July 28, 2013 10:10 PM
  • check out this blog, it might be helpful to your issue:

    http://blogs.technet.com/b/sbs/archive/2009/01/29/cannot-resolve-names-in-certain-top-level-domains-like-co-uk.aspx


    Every second counts..make use of it.

    Monday, July 29, 2013 7:21 AM
  • Thanks for your efforts Ace. Think I found the doc you meant, though unfortunately while they mention limits they don't specify what they are, and they relate to WHOIS query limits rather than DNS query limits. It was probably the WHOIS side of thing I was thinking of.

    Thanks Cguan, that might well explain it, and other than the suggestion of it happening immediately (I only see it occasionally) that explanation could well fit with what I've been seeing. Would certainly fit with me never seeing it on both resolvers at the same time, since I obviously never restart both servers at the same time, so only one of them would hit the ttl timeout at the same time.

    Most annoying thing... I recognise that blog post! Think I read it a while ago when looking for something else, ignored it as it didn't resolve the issue I was seeing, and obviously this long running issue didn't come to mind!

    Monday, July 29, 2013 5:56 PM
  • That blog is 4 years old. There have been hotfixes to address that issue. Also, if you believe it's a TTL issue, then it may be based on certain domain names that have CNAMES with a longer TTL than the A record's TTL. You can find out the differences and compare them using DIG.

    Here's more on the hotfixes. Run the following, and if the hotfix is already installed or it doesn't apply due to service pack level or operating system version, no fret, the installer will tell you right away. Some of them require restarts.

    DNS Server service does not use root hints to resolve external names in Windows Server 2008 R2
    Post Windows 2008 R2 SP1 HOTFIX available.
    APPLIES TO •Windows 2008 R2 Datacenter •Windows 2008 R2 Ent •Windows 2008 R2 Std.
    Requires a restart.
    http://support.microsoft.com/kb/2616776

    Windows 2008 -
    DNS queries for external domains are not resolved when you use Conditional Forwarding in Windows Server 2008
    Post Windows 2008 SP2 Hotfix available
    Requires a restart.
    http://support.microsoft.com/kb/2625735/
     
    DNS server stops responding to DNS queries from client computers in in Windows Server 2003, in Windows Server 2008 or in Windows Server 2008 R2 - Post Service Pack Hotfix available.
    Does not require a restart.
    http://support.microsoft.com/kb/2655960
     
    DNS Server service does not resolve some external DNS names after it works for a while in Windows Server 2008 R2
    Hotfix release - (released 4/15/2011)
    http://support.microsoft.com/kb/2508835

    And if nslookup times out on MX records, it's by design:
    NSLOOKUP Returns Time-out Error When Query for an MX Record
    http://support.microsoft.com/kb/198551/en-us
    ===

    -

    Using DIG:

    ==================================================================
    More info on CNAME TTL issues and examples of CNAME TTL and record TTL discrepancies using cdc.gov as an example.


    www.cdc.gov is a CNAME pointing to www.cdc.gov.edgesuite.net that has a 21,600 seconds TTL (6 hours).

    You can see the different TTLs (www.cdc.gov is 3600, yet www.cdc.gov.edgesuite.net is 21,600):
    (Results from: http://www.domaintools.com/research/dns/?query=www.cdc.gov&search=dns

    You can see the different TTLs (www.cdc.gov is 3600, yet www.cdc.gov.edgesuite.net is 21,600):
    (Results from: http://www.domaintools.com/research/dns/?query=www.cdc.gov&search=dns

    Www.cdc.gov DNS Lookup
    ;; Answer received from 127.0.0.1 (291 bytes)
    ;;
    ;; HEADER SECTION
    ;; id = 61515
    ;; qr = 1 opcode = QUERY aa = 0 tc = 0 rd = 1
    ;; ra = 1 ad = 0 cd = 0 rcode = NOERROR
    ;; qdcount = 1 ancount = 4 nscount = 9 arcount = 0

    ;; QUESTION SECTION (1 record)
    ;; www.cdc.gov. IN A

    ;; ANSWER SECTION (4 records)
    www.cdc.gov. 3600 IN CNAME www.cdc.gov.edgesuite.net.
    www.cdc.gov.edgesuite.net. 21600 IN CNAME a1433.g.akamai.net.
    a1433.g.akamai.net. 20 IN A 216.243.20.9
    a1433.g.akamai.net. 20 IN A 216.243.20.17

    ;; AUTHORITY SECTION (9 records)
    g.akamai.net. 1800 IN NS n2g.akamai.net.
    g.akamai.net. 1800 IN NS n3g.akamai.net.
    g.akamai.net. 1800 IN NS n7g.akamai.net.
    g.akamai.net. 1800 IN NS n6g.akamai.net.
    g.akamai.net. 1800 IN NS n8g.akamai.net.
    g.akamai.net. 1800 IN NS n1g.akamai.net.
    g.akamai.net. 1800 IN NS n5g.akamai.net.
    g.akamai.net. 1800 IN NS n4g.akamai.net.
    g.akamai.net. 1800 IN NS n0g.akamai.net.

    ;; ADDITIONAL SECTION (0 records)


    Another example is www.techdirt.com:
    Results from http://www.domaintools.com/research/dns/?query=www.techdirt.com&search=dns

    Www.techdirt.com DNS Lookup
    ;; Answer received from 127.0.0.1 (189 bytes)
    ;;
    ;; HEADER SECTION
    ;; id = 23992
    ;; qr = 1 opcode = QUERY aa = 0 tc = 0 rd = 1
    ;; ra = 1 ad = 0 cd = 0 rcode = NOERROR
    ;; qdcount = 1 ancount = 5 nscount = 2 arcount = 2

    ;; QUESTION SECTION (1 record)
    ;; www.techdirt.com. IN A

    ;; ANSWER SECTION (5 records)
    www.techdirt.com. 172800 IN CNAME techdirt.com.
    techdirt.com. 900 IN A 208.53.48.36
    techdirt.com. 900 IN A 208.53.48.129
    techdirt.com. 900 IN A 208.53.48.153
    techdirt.com. 900 IN A 208.53.48.33

    ;; AUTHORITY SECTION (2 records)
    techdirt.com. 172800 IN NS ns.dnsbox.net.
    techdirt.com. 172800 IN NS ns2.dnsbox.net.

    ;; ADDITIONAL SECTION (2 records)
    ns.dnsbox.net. 900 IN A 208.53.48.154
    ns2.dnsbox.net. 900 IN A 67.227.171.37

    -

    The point is I didn't think of taking it in this direction, since you were focused on .info domains. Your folks may be using .info more than others, so you may not have seen this with other TLDs or domains, in general.

    -

    EDNS0:

    Another thing to take a look at is EDNS0, which can be a big factor and can also cause this. Hopefully your firewall supports it and you haven't disabled it on any machine.

    Here's a quick command to test if there's an EDNS0 restriction in your firewall:
    nslookup -type=TXT rs.dns-oarc.net

    Or if you want to test a specific DNS server for EDNS0 support, whether an internal or external DNS server, use the following method:

    c:\>nslookup
    > server 4.2.2.2    <---- change the IP to whatever DNS server you want to test for EDNS0 support
    > set q=txt
    > rs.dns-oarc.net

    FYI: In the nslookup results, look for the two parts in the response that say, " ...DNS reply size limit is at least xxxx." The xxxx is the DNS UDP packet size that your firewall or forwarder will support, assuming EDNS0 has not been disabled on the DNS server. If it's under 512, then that DNS

    doesn't support it, or the firewall doesn't support it and is blocking EDNS0 or the Forwarder you are using is blocking or not allowing/configured to use EDNS0.

    More on this in my blog:

    What is EDNS0? (Extension mechanisms for DNS)
    Published by Ace Fekay, MCT, MVP DS on Oct 11, 2010 at 2:46 PM
    http://msmvps.com/blogs/acefekay/archive/2010/10/11/edns0-extension-mechanisms-for-dns.aspx


    Ace Fekay
    MVP, MCT, MCITP/EA, MCTS Windows 2008/R2 & Exchange 2007, Exchange 2010 EA, MCSE & MCSA 2003/2000, MCSA Messaging 2003
    Microsoft Certified Trainer
    Microsoft MVP - Directory Services
    Technical Blogs & Videos: http://www.delawarecountycomputerconsulting.com/

    This post is provided AS-IS with no warranties or guarantees and confers no rights.

    FaceBook Twitter LinkedIn

    Monday, July 29, 2013 7:15 PM
  • Should have known it wouldn't be that simple! :-) Thanks for all the info Ace, I'll go through it all and try your suggestions when I'm at work tomorrow.
    Monday, July 29, 2013 7:41 PM
  • Nothing is simple in this business, despite what the boss or end users, usually one in the same, believe. :-)

    -

    And one more thing to muddy the issue. It could be a DDOS or DNS amplification attack. There have been many reports lately concerning this as well as discussions. It could be their DNS servers are being attacked when you see the symptoms, especially if it occurs trying some other resolvers such as what I mentioned. It all goes back to the .info registrar.

    If you can at least cover your basis above with the hotfixes, updates, etc, you'll know you'll be ok. As for attacks on your registrar's nameservers, that is beyond your control. Here's where this was discussed:

    Thread: "Protecting Windows DNS Server from being abused for DNS amplification attacks" 4/10/2013
    http://social.technet.microsoft.com/Forums/en-US/winserverNIS/thread/fac86dc7-779d-48eb-a113-9c06c2222af9

    DNSSEC and DNS Amplification Attacks - Explanation of this DDOS attack, and suggestions to prevent an attack.
    By Greg Lindsay, MSFT, 4/23/2013
    http://technet.microsoft.com/en-us/security/hh972393.aspx

    Thread: "How to filter isc.org ANY attacks (DNS Amplification Attack)" 11/6/2012
    http://social.technet.microsoft.com/Forums/en-US/Forefrontedgegeneral/thread/8a9d1c5e-a2df-4136-98cc-051fcde69f55

    Excellent discussion:
    Thread: "DDoS on Server 2008 R2," 6/3/2013
    http://social.technet.microsoft.com/Forums/windowsserver/en-US/74125383-dad6-4a60-af0c-471849af6dc2/ddos-on-server-2008-r2


    Ace Fekay
    MVP, MCT, MCITP/EA, MCTS Windows 2008/R2 & Exchange 2007, Exchange 2010 EA, MCSE & MCSA 2003/2000, MCSA Messaging 2003
    Microsoft Certified Trainer
    Microsoft MVP - Directory Services
    Technical Blogs & Videos: http://www.delawarecountycomputerconsulting.com/

    This post is provided AS-IS with no warranties or guarantees and confers no rights.

    FaceBook Twitter LinkedIn

    Monday, July 29, 2013 9:00 PM