locked
Windows Server 2008 R2 DNS DNS error 5504 on some queries (EDNS0 problem?) RRS feed

  • Question

  • I just migrated my AD servers from Win 2008 (Ent Ger x64) to Win 2008R2 (Ent Ger x64).

    After that, some dns queries failed and DNS Error 5504 "The DNS Server encountered an invalid domain name in a packet from X.X.X.X ..." were loged.

    e.g.
    "nslookup www.microsoft.com"
    You'll get 7 5504 log entries; one for each akadns.net Name server (www.microsoft.com is a CNAME of toggle.www.ms.akadns.net)

    I found this article http://www.simpledns.com/kb.aspx?kbid=1239 "Why do I get "Header: Format Error" responses from Akamai DNS servers?"  
    I  changed the dns config with "dnscmd /config /enableednsprobes 0" and then 1 but no success

    Workaround: using a dns forwarder

    any suggestions?
    regards
    Frank
    Friday, August 21, 2009 3:58 PM

Answers

  • Hi Necati,

    I think, we got it and maybe come back to the ISA issue, when TMG is RTM.

    Thank You

    Symptom:
    Upgrading Windows server 2008 to 2008 R2 changed the behavior of the DNS servers behind a firewall (e.g. MS ISA). DNS queries to certain domains fail and DNS event 5504 is logged.

    Solution:
    Follow http://support.microsoft.com/kb/828263/en-us “DNS query responses do not travel through a firewall in Windows Server 2003” and Turn off EDNS0 functionality with
    dnscmd /config /enableednsprobes 0

    kind regards
    Frank

    • Marked as answer by Frank Warius Saturday, August 22, 2009 11:52 AM
    Saturday, August 22, 2009 11:52 AM

All replies

  • Hi Frank,

    This looks like Cache Polution protection blocking NS or A responses from nonauth servers for that domain. Can you please put a simple network trace of your query and response that's causing this event.
    Best Regards Necati Cehreli - CCNA,CCNP,CCIP,MCTS,MCITP http://ncehreli.blogspot.com
    Friday, August 21, 2009 7:24 PM
  • Hi Necati,

    thank You for the hint about the network traffic.

    I had a look on my firewall (ISA) and found this
     
    Initiated Connection WRS-SRV19 22.08.2009 08:22:44
    Log type: Firewall service
    Status: Der Vorgang wurde erfolgreich beendet. 
    Rule: Web Infra
    Source: Internal (192.168.151.1:55857)
    Destination: External (193.108.91.2:53)
    Protocol: DNS
     Additional information
    Number of bytes sent: 0 Number of bytes received: 0
    Processing time: 0ms Original Client IP: 192.168.151.1
     
    Closed Connection WRS-SRV19 22.08.2009 08:24:14
    Log type: Firewall service
    Status: A connection was gracefully closed in an orderly shutdown process with a three-way FIN-initiated handshake
    Rule: Web Infra
    Source: Internal (192.168.151.1:55857)
    Destination: External (193.108.91.2:53)
    Protocol: DNS
     Additional information
    Number of bytes sent: 61 Number of bytes received: 12
    Processing time: 90078ms Original Client IP: 192.168.151.1
     
    this leeds to http://support.microsoft.com/?id=828263 "DNS query responses do not travel through a firewall in Windows Server 2003" and we have another workaround

    workaround 2: "dnscmd /config /enableednsprobes 0" (without switching it on afterwards)

    So an ISA (TMG latest Beta) problem is left: how to increase the UDP packet size over 512 B

    Than You
    regards
    Frank
    Saturday, August 22, 2009 7:17 AM
  • Hi Frank

    There is no known method (at least by me) for tweaking Oversized UDP Packets on Microsoft ISA. On Checkpoint NGX from WebIntelligence you can disable DNS Policy Enforcements to overcome this issue and also Cisco has a command for this. But keep in mind that oversized UDP packets can have negative effects like opening your system to DoS attacks especially on AD DS environments.

    If the workaround in the technet article is not sufficient for you you can check on the ISA forums for more solutions.

    mit freundliche Grüße

    Best Regards Necati Cehreli - CCNA,CCNP,CCIP,MCTS,MCITP http://ncehreli.blogspot.com
    Saturday, August 22, 2009 9:24 AM
  • Hi Necati,

    I think, we got it and maybe come back to the ISA issue, when TMG is RTM.

    Thank You

    Symptom:
    Upgrading Windows server 2008 to 2008 R2 changed the behavior of the DNS servers behind a firewall (e.g. MS ISA). DNS queries to certain domains fail and DNS event 5504 is logged.

    Solution:
    Follow http://support.microsoft.com/kb/828263/en-us “DNS query responses do not travel through a firewall in Windows Server 2003” and Turn off EDNS0 functionality with
    dnscmd /config /enableednsprobes 0

    kind regards
    Frank

    • Marked as answer by Frank Warius Saturday, August 22, 2009 11:52 AM
    Saturday, August 22, 2009 11:52 AM