none
Windows 2008 R2 DNS Query Not Retrieving all Records

    Question

  • Windows 2008 R2 SP1, fully patched.

    If I do a query for products.verizonwireless.com on any of my 2008 R2 DNS servers it fails, however works on plain 2008 or 2003 servers DNS servers. The correct results should be like below.

    products.verizonwireless.com. NS 900 ns0.moxieinteractive.net.
    products.verizonwireless.com. NS 900 ns1.moxieinteractive.net.
    products.verizonwireless.com. CNAME 300 products.verizonwireless.com.edgekey.net

    However on the 2008 R2 it doesn't retrieve the CNAME record so resolution fails. It's not an issue with root hints or replication since I can do nslookup -type=any products.verizonwireless.com and it will get all 3 records. It's something with the DNS engine that won't get it it. What I've tried:

    1. Clear DNS server cache and local cache

    2. cmd /config /EnableEDNSProbes 0 (Was already set months ago when we deployed and still set that way)

    3. Tried nsloookup

    set vc (to have nslookup use tcp instead of udp to test if due to 512 kb limitation) query still fails to retreive the cname

    4. Fully patched the machine.

    5. Enabled debugging on the non R2 and R2, logs show resolution going exact same but the same verizon server only gives 2 records the R2 server but 3 to the non R2 with no errors.

    Can someone with an R2 DNS do a lookup for products.verizonwireless.com and see if this a bug? I had a colleage try and he was getting the same result. I'm thinking it's isolated to just R2 boxes.


    James Chong MCITP | EA | EMA; MCSE | M+, S+ Security+, Project+, ITIL msexchangetips.blogspot.com
    Tuesday, July 19, 2011 7:17 PM

Answers

  • Windows 2008 R2 SP1, fully patched.

    If I do a query for products.verizonwireless.com on any of my 2008 R2 DNS servers it fails, however works on plain 2008 or 2003 servers DNS servers. The correct results should be like below.

    products.verizonwireless.com. NS 900 ns0.moxieinteractive.net.
    products.verizonwireless.com. NS 900 ns1.moxieinteractive.net.
    products.verizonwireless.com. CNAME 300 products.verizonwireless.com.edgekey.net

    This is not an R2 "bug" but ... a "feature" :) - See, the problem is caused by the fact that the DNS records for that host are totally screwed; a lookup for the host returns the following

    products.verizonwireless.com. 299 IN    CNAME   products.verizonwireless.com.edgekey.net.
    products.verizonwireless.com.edgekey.net. 21280 IN CNAME e3516.c.akamaiedge.net.
    e3516.c.akamaiedge.net. 19      IN      A       95.100.158.87

    now, the RFCs related to DNS mandate that a CNAME record can't point to another CNAME so, whoever configured that record probably was totally clueless regarding DNS; your DNS resolver is doing exactly what it is expected to do, that is discarding those answers also since they may, in some cases, be used to leverage some very bad tricks (e.g. DNS poisoning), the correct way of setting up those DNS records would have been

    products.verizonwireless.com. 299 IN    CNAME   e3516.c.akamaiedge.net.
    products.verizonwireless.com.edgekey.net. 21280 IN CNAME e3516.c.akamaiedge.net.
    e3516.c.akamaiedge.net. 19      IN      A       95.100.158.87


    That is, pointing both CNAMEs to the same A record

    As for the fact that you disabled EDNS, I don't think it's a good idea; see, DNSSEC relies on EDNS to work and lacking EDNS support may (and will) cause you a lot of resolution issues, so, instead of disabling EDNS, try to troubleshoot it by following this and ensure it's working as expected; the idea of disabling EDNS support may be ok while trying to troubleshoot an issue but it isn't a good idea to keep it disabled

     


    • Marked as answer by Tiger Li Wednesday, July 20, 2011 8:48 AM
    • Edited by ObiWan Wednesday, July 20, 2011 3:09 PM reformatted text
    Wednesday, July 20, 2011 8:38 AM

All replies

  • Which internet public DNS server is this windows server 2008 R2 sp1 host pointing to use? Have you tested by using other public DNS servers like 8.8.8.8?

    Wednesday, July 20, 2011 5:32 AM
  • Windows 2008 R2 SP1, fully patched.

    If I do a query for products.verizonwireless.com on any of my 2008 R2 DNS servers it fails, however works on plain 2008 or 2003 servers DNS servers. The correct results should be like below.

    products.verizonwireless.com. NS 900 ns0.moxieinteractive.net.
    products.verizonwireless.com. NS 900 ns1.moxieinteractive.net.
    products.verizonwireless.com. CNAME 300 products.verizonwireless.com.edgekey.net

    This is not an R2 "bug" but ... a "feature" :) - See, the problem is caused by the fact that the DNS records for that host are totally screwed; a lookup for the host returns the following

    products.verizonwireless.com. 299 IN    CNAME   products.verizonwireless.com.edgekey.net.
    products.verizonwireless.com.edgekey.net. 21280 IN CNAME e3516.c.akamaiedge.net.
    e3516.c.akamaiedge.net. 19      IN      A       95.100.158.87

    now, the RFCs related to DNS mandate that a CNAME record can't point to another CNAME so, whoever configured that record probably was totally clueless regarding DNS; your DNS resolver is doing exactly what it is expected to do, that is discarding those answers also since they may, in some cases, be used to leverage some very bad tricks (e.g. DNS poisoning), the correct way of setting up those DNS records would have been

    products.verizonwireless.com. 299 IN    CNAME   e3516.c.akamaiedge.net.
    products.verizonwireless.com.edgekey.net. 21280 IN CNAME e3516.c.akamaiedge.net.
    e3516.c.akamaiedge.net. 19      IN      A       95.100.158.87


    That is, pointing both CNAMEs to the same A record

    As for the fact that you disabled EDNS, I don't think it's a good idea; see, DNSSEC relies on EDNS to work and lacking EDNS support may (and will) cause you a lot of resolution issues, so, instead of disabling EDNS, try to troubleshoot it by following this and ensure it's working as expected; the idea of disabling EDNS support may be ok while trying to troubleshoot an issue but it isn't a good idea to keep it disabled

     


    • Marked as answer by Tiger Li Wednesday, July 20, 2011 8:48 AM
    • Edited by ObiWan Wednesday, July 20, 2011 3:09 PM reformatted text
    Wednesday, July 20, 2011 8:38 AM
  •  

    Which internet public DNS server is this windows server 2008 R2 sp1 host pointing to use? Have you tested by using other public DNS servers like 8.8.8.8?


    First of all... why, in the name of the Good Lord, a DNS server should be pointing to some "public DNS servers" when it's perfectly able to carry out the whole resolution process by itself using root-hints and recursion ?

    Then, using either the google DNS servers or the OpenDNS servers as forwarders isn't a good idea; first of all they don't support EDNS (their packet size limit is 512), then, they "play tricks" with DNS responses and this may totally break things, especially if you're running a mailserver

    Bottom line, avoid using forwarders or, in any case, use them only if really, strictly needed (e.g. a remote office forwarding DNS queries to the central office DNS), instead, understand how the DNS resolution work and configure your DNS to use root-hints and recursion; also keep in mind that replies returned by an external DNS used as a forwarders will be accepted and trusted by our DNS, so, if the external DNS gets poisoned or compromised, the issue will in turn affect our DNS

     


    Wednesday, July 20, 2011 9:01 AM
  • Thanks for the great explanation! That makes sense when I was looking at the debug logs and comparing I was thinking along the lines of R2 not wanting to do the redirect ie. resolves to cname then resolver does another looking for that cname. But your explanation puts it together.

    One question what is the exact feature of R2 that is blocking this behavior since it works with other OS? Also even though non rfc compliant the fact that other OSs and and other third party DNS servers can resolve correctly how is MS only making R2 enforce this?


    James Chong MCITP | EA | EMA; MCSE | M+, S+ Security+, Project+, ITIL msexchangetips.blogspot.com
    Wednesday, July 20, 2011 2:02 PM
  • Thanks for the great explanation! That makes sense when I was looking at the debug logs and comparing I was thinking along the lines of R2 not wanting to do the redirect ie. resolves to cname then resolver does another looking for that cname. But your explanation puts it together.

    One question what is the exact feature of R2 that is blocking this behavior since it works with other OS? Also even though non rfc compliant the fact that other OSs and and other third party DNS servers can resolve correctly how is MS only making R2 enforce this?

    Thank you for the feedback and... nothing special, really, I routinely deal with DNS, networking and other stuff, so it was just a matter of running a couple of DNS queries

    As for the "feature"... I know what's causing such a behaviour, but I WON'T tell you :) ! Seriously, the DNS is working as it's expected to work and sincerely I don't think that disabling such a check would be a good idea; if the folks at verizon screwed their DNS, let them go, sooner or later a bunch of people will start complaining with them and they'll fix the DNS :)

    See, the "CNAME to CNAME" redirection was supported by the ancient BIND v4 but then, since it was a deprecated feature (for a bunch of reasons, I won't fathom the whole story here), it was removed ... now, the Microsoft DNS was basically born from BIND v4, although, as of today, it's a totally different kind of critter (thanks Microsoft :D) and the fact that it's now correctly protecting its cache from poisoning (hint, hint) is really good; I just wonder why 2003 doesn't the same... oh well

     

    Wednesday, July 20, 2011 2:14 PM
  • Thanks again!
    James Chong MCITP | EA | EMA; MCSE | M+, S+ Security+, Project+, ITIL msexchangetips.blogspot.com
    Wednesday, July 20, 2011 2:51 PM
  • Obi, that was a great explanation. I saw this too late to respond, because well, you covered it all! :-) 

    Cheers!

     


    Ace Fekay
    MVP, MCT, MCITP EA, MCTS Windows 2008 & Exchange 2007 & Exchange 2010, Exchange 2010 Enterprise Administrator, MCSE & MCSA 2003/2000, MCSA Messaging 2003
    Microsoft Certified Trainer
    Microsoft MVP - Directory Services
    Complete List of Technical Blogs: http://www.delawarecountycomputerconsulting.com/technicalblogs.php

    This posting is provided AS-IS with no warranties or guarantees and confers no rights.

    Thursday, July 21, 2011 2:19 AM
  • Thinking about this after I posted, in addition to Verizon incorrectly configuring their records (excellent point on that, Obi), I believe it may *also* comes down to a CNAME vs RR TTL difference, especially if one CNAME points to another with a different TTL, and the A record has a different TTL. The question comes down to, which one gets cached? If the A record expires before the CNAME does, it won't re-look it up until the CNAME expires, which causes problems. There is another one I am aware of that's misconfigured -> www.cdc.gov, but they may have fixed it. I didn't look it up to check, but about 6 months ago, it was generated complaints with 2008 R2 for the same reasons.

    Look at the different TTLs in the results towards the bottom of this post for products.verizonwireless.com. This can be alleviated using the MaxCacheTTL reg entry mentioned in the following links, but I think why do we need to alter a DNS server because of misconfigured records?


    Cannot resolve names in certain top level domains like .co.uk.
    http://blogs.technet.com/b/sbs/archive/2009/01/29/cannot-resolve-names-in-certain-top-level-domains-like-co-uk.aspx

    Windows Server 2008 DNS Servers may fail to resolve queries for some top-level domains: 
    http://support.microsoft.com/?id=968372

     

    In addition, here's more info and a couple of threads on the subject, if you would like to read up on it.

    DNS Server service does not resolve some external DNS names after it works for a while in Windows Server 2008 R2 (released 4/15/2011)
    http://support.microsoft.com/kb/2508835

    Technet Forum Thread: CNAME caching problem on Windows Server 2003 R2:
    http://social.technet.microsoft.com/Forums/en-US/winserverNIS/thread/97dcc9cd-76c3-4813-80a0-4173c0030319

    Technet Forum Thread: Windows 2008 DNS caching being wonky
    http://social.technet.microsoft.com/Forums/en-US/winserverNIS/thread/a4e58fdf-fb3f-451e-b85a-247365df6b8d/

    CNAME and A record have different TTLs. Which one will be cached?
    http://serverfault.com/questions/123663/cname-and-a-record-have-different-ttls-which-one-will-be-cached

     

     

    Lookup results for products.verizonwireless.com. You can clearly see the mismatched TTLS.

    ========================================
    ========================================
    And the following was from an nslookup with the D2 option:

    ------------
    ------------
    Got answer (149 bytes):
        HEADER:
            opcode = QUERY, id = 4, rcode = NOERROR
            header flags:  response, want recursion, recursion avail.
            questions = 1,  answers = 3,  authority records = 0,  additional = 0

        QUESTIONS:
            Products.verizonwireless.com, type = A, class = IN
        ANSWERS:
        ->  Products.verizonwireless.com
            type = CNAME, class = IN, dlen = 42
            canonical name = products.verizonwireless.com.edgekey.net
            ttl = 300 (5 mins)
        ->  products.verizonwireless.com.edgekey.net
            type = CNAME, class = IN, dlen = 21
            canonical name = e3516.c.akamaiedge.net
            ttl = 13824 (3 hours 50 mins 24 secs)
        ->  e3516.c.akamaiedge.net
            type = A, class = IN, dlen = 4
            internet address = 184.50.46.87
            ttl = 20 (20 secs)
    =================================

     

     

    The following results came from:
    http://www.domaintools.com/research/dns/?query=products.verizonwireless.com&search=dns
    ========================================
    ========================================
    ;; Answer received from 127.0.0.1 (311 bytes)
    ;;
    ;; HEADER SECTION
    ;; id = 31917
    ;; qr = 1 opcode = QUERY aa = 0 tc = 0 rd = 1
    ;; ra = 1 ad = 0 cd = 0 rcode = NOERROR
    ;; qdcount = 1 ancount = 3 nscount = 9 arcount = 0

    ;; QUESTION SECTION (1 record)
    ;; products.verizonwireless.com. IN A

    ;; ANSWER SECTION (3 records)
    products.verizonwireless.com. 300 IN CNAME products.verizonwireless.com.edgekey.net.
    products.verizonwireless.com.edgekey.net. 21600 IN CNAME e3516.c.akamaiedge.net.
    e3516.c.akamaiedge.net. 20 IN A 69.192.206.87

    ;; AUTHORITY SECTION (9 records)
    c.akamaiedge.net. 1800 IN NS n0c.akamaiedge.net.
    c.akamaiedge.net. 1800 IN NS n4c.akamaiedge.net.
    c.akamaiedge.net. 1800 IN NS n2c.akamaiedge.net.
    c.akamaiedge.net. 1800 IN NS n8c.akamaiedge.net.
    c.akamaiedge.net. 1800 IN NS n7c.akamaiedge.net.
    c.akamaiedge.net. 1800 IN NS n6c.akamaiedge.net.
    c.akamaiedge.net. 1800 IN NS n5c.akamaiedge.net.
    c.akamaiedge.net. 1800 IN NS n3c.akamaiedge.net.
    c.akamaiedge.net. 1800 IN NS n1c.akamaiedge.net.

    ;; ADDITIONAL SECTION (0 records)

     

    Ace

    Late edit: I had to propose this as an additional answer to what is going on with Verizon's domain records due to the CNAME vs RR differences in conjunction with Obi's observations regarding CNAME pointing to another CNAME undesireable configuration.


    Ace Fekay
    MVP, MCT, MCITP EA, MCTS Windows 2008 & Exchange 2007 & Exchange 2010, Exchange 2010 Enterprise Administrator, MCSE & MCSA 2003/2000, MCSA Messaging 2003
    Microsoft Certified Trainer
    Microsoft MVP - Directory Services
    Complete List of Technical Blogs: http://www.delawarecountycomputerconsulting.com/technicalblogs.php

    This posting is provided AS-IS with no warranties or guarantees and confers no rights.


    Thursday, July 21, 2011 2:43 AM
  • Thinking about this after I posted, in addition to Verizon incorrectly configuring their records (excellent point on that, Obi), I believe it may *also* comes down to a CNAME vs RR TTL difference, especially if one CNAME points to another with a different TTL, and the A record has a different TTL. The question comes down to, which one gets cached? If the A record expires before the CNAME does, it won't re-look it up until the CNAME expires, which causes problems. There is another one I am aware of that's misconfigured -> www.cdc.gov, but they may have fixed it. I didn't look it up to check, but about 6 months ago, it was generated complaints with 2008 R2 for the same reasons.

    Look at the different TTLs in the results towards the bottom of this post for products.verizonwireless.com. This can be alleviated using the MaxCacheTTL reg entry mentioned in the following links, but I think why do we need to alter a DNS server because of misconfigured records?


    All good points Ace (I overlooked the TTL... I saw that double CNAME and didn't look elsewhere :D!), and... I'd avoid changing the DNS config, that may expose the DNS to cache poisoning and sincerely, lowering the fence just to allow working with a badly configured zone doesn't seem a good idea to me :)


    Thursday, July 21, 2011 7:09 AM
  • All good points Ace (I overlooked the TTL... I saw that double CNAME and didn't look elsewhere :D!), and... I'd avoid changing the DNS config, that may expose the DNS to cache poisoning and sincerely, lowering the fence just to allow working with a badly configured zone doesn't seem a good idea to me :)

    I've seen this come up occassionally with some records. Unfortunate that some major entities have misconfigured records not realizing the impact it causes accessing their websites or other resources.

    Ace


    Ace Fekay
    MVP, MCT, MCITP EA, MCTS Windows 2008 & Exchange 2007 & Exchange 2010, Exchange 2010 Enterprise Administrator, MCSE & MCSA 2003/2000, MCSA Messaging 2003
    Microsoft Certified Trainer
    Microsoft MVP - Directory Services
    Complete List of Technical Blogs: http://www.delawarecountycomputerconsulting.com/technicalblogs.php

    This posting is provided AS-IS with no warranties or guarantees and confers no rights.

    Thursday, July 21, 2011 3:22 PM
  • I've seen this come up occassionally with some records.
    Unfortunate that some major entities have misconfigured
    records not realizing the impact it causes accessing their
    websites or other resources.

    Yes; in this case the wrong DNS entry wasn't sitting on some obscure domain, but on the Verizon one and ... well, one would expect that  folks at Verizon know how to correctly setup their DNS zones... oh well :) !!


    • Edited by ObiWan Thursday, July 21, 2011 4:09 PM
    Thursday, July 21, 2011 3:42 PM
  • Yes; in this case the wrong DNS entry wasn't sitting on some obscure
    domain, but on the Verizon one and ... well, one would expect that folks
    at Verizon know how to correctly setup their DNS zones... oh well :) !!


    Agreed!

    Ace Fekay
    MVP, MCT, MCITP EA, MCTS Windows 2008 & Exchange 2007 & Exchange 2010, Exchange 2010 Enterprise Administrator, MCSE & MCSA 2003/2000, MCSA Messaging 2003
    Microsoft Certified Trainer
    Microsoft MVP - Directory Services
    Complete List of Technical Blogs: http://www.delawarecountycomputerconsulting.com/technicalblogs.php

    This posting is provided AS-IS with no warranties or guarantees and confers no rights.

    Thursday, July 21, 2011 3:53 PM
  • Thanks for the great explanation! That makes sense when I was looking at the debug logs and comparing I was thinking along the lines of R2 not wanting to do the redirect ie. resolves to cname then resolver does another looking for that cname. But your explanation puts it together.

    One question what is the exact feature of R2 that is blocking this behavior since it works with other OS? Also even though non rfc compliant the fact that other OSs and and other third party DNS servers can resolve correctly how is MS only making R2 enforce this?

    Thank you for the feedback and... nothing special, really, I routinely deal with DNS, networking and other stuff, so it was just a matter of running a couple of DNS queries

    As for the "feature"... I know what's causing such a behaviour, but I WON'T tell you :) ! Seriously, the DNS is working as it's expected to work and sincerely I don't think that disabling such a check would be a good idea; if the folks at verizon screwed their DNS, let them go, sooner or later a bunch of people will start complaining with them and they'll fix the DNS :)

    See, the "CNAME to CNAME" redirection was supported by the ancient BIND v4 but then, since it was a deprecated feature (for a bunch of reasons, I won't fathom the whole story here), it was removed ... now, the Microsoft DNS was basically born from BIND v4, although, as of today, it's a totally different kind of critter (thanks Microsoft :D) and the fact that it's now correctly protecting its cache from poisoning (hint, hint) is really good; I just wonder why 2003 doesn't the same... oh well

    Obi,

    If you're still following this thread, if you have a moment, check out the following thread, if you have anything to offer regarding this same subject in this thread:
    http://social.technet.microsoft.com/Forums/en-US/winserverDS/thread/e4a97a9b-cb1d-43f1-aa5b-1abb34bddfa5/

    .

    Thanks,
    Ace


    Ace Fekay
    MVP, MCT, MCITP Enterprise Administrator, MCTS Windows 2008 & Exchange 2007 & Exchange 2010, Exchange 2010 Enterprise Administrator, MCSE & MCSA 2003/2000, MCSA Messaging 2003
    Microsoft Certified Trainer
    Microsoft MVP - Directory Services
    Complete List of Technical Blogs: http://www.delawarecountycomputerconsulting.com/technicalblogs.php

    This posting is provided AS-IS with no warranties or guarantees and confers no rights.

    FaceBook Twitter LinkedIn

    Thursday, February 23, 2012 2:00 AM
  • Ace ... I'm opening it right now :)

    Uh... and... in case of need, my email works, just use my "mvps.org" address ;-)
    • Edited by ObiWan Tuesday, February 28, 2012 9:17 AM
    Tuesday, February 28, 2012 9:17 AM
  • Ace ... I'm opening it right now :)

    Uh... and... in case of need, my email works, just use my "mvps.org" address ;-)

    Thanks, and forgot! I was juggling these two threads and thought you would get this quickly. :-)

    As well as that others will have a reference to that other thread, too!


    Ace Fekay
    MVP, MCT, MCITP Enterprise Administrator, MCTS Windows 2008 & Exchange 2007 & Exchange 2010, Exchange 2010 Enterprise Administrator, MCSE & MCSA 2003/2000, MCSA Messaging 2003
    Microsoft Certified Trainer
    Microsoft MVP - Directory Services
    Complete List of Technical Blogs: http://www.delawarecountycomputerconsulting.com/technicalblogs.php

    This posting is provided AS-IS with no warranties or guarantees and confers no rights.

    FaceBook Twitter LinkedIn

    Thursday, March 01, 2012 12:43 AM
  • Thanks, and forgot! I was juggling these two threads and thought you would get this quickly. :-)

    As well as that others will have a reference to that other thread, too!

    Well... just call me and I'll come :) (as a note, does YOUR mvps account work ? I sent you an "e" ...)

    As for the threads, hope my short message was of help to shed some more light upon that "issue" in the other thread, although you were perfectly handling it (as usual, by the way :D)

    Thursday, March 01, 2012 7:26 AM