none
Internal AD DNS server outage RRS feed

  • Question

  • We had a internal DNS outage at my job about a week ago. All of our internal applications pointing to our 3 servers couldn't find their A records and failed accross the board. This lasted about 30-45 minutes and was resolved after restarting the DNS services.

    I opened a ticket with Microsoft to determine what happened but they haven't really be able to tell me anything other than "your internal network probably went down" - which of course, did not happen. We have reason to believe that the Administrator in charge of AD and internal DNS made an unauthorized change which led to the outage. 

    I've been investigating the registry, file system, and logs trying to track down why exactly internal DNS went down. So far, I have had no luck in finding any sort of concrete evidence. My question is - where and what should I be looking at to see if our Admin made some unauthorized changes? Event log shows no failures during the time frame. I actually see the event for the service being restarted on 1 of our 3 servers only which leads be to believe the changes most likely occurred on that server. Any help would be appreciated. 

    One other note - another one of our admins working around the issue at the time did tell me that a certain type of entry was made and then quickly deleted after it was not the correct way to make the entry. Hoping this information might ring a bell. 

    Tuesday, May 3, 2016 2:39 PM

Answers

  • If there are no events logged then I do not see another way to figure out happened. In similar scenarios, I apply what is mentioned in the Wiki I started here: http://social.technet.microsoft.com/wiki/contents/articles/24170.how-to-provide-temporary-and-secure-administrative-access-to-critical-systems-and-applications.aspx

    It requires Orchestrator and it allows me to keep an eye on what people are doing. In addition, admins have no admin rights by default, they need to request for them through the self service portal. This may inspire you to have a similar control in place.


    This posting is provided AS IS with no warranties or guarantees , and confers no rights.

    Ahmed MALEK

    My Website Link

    My Linkedin Profile

    My MVP Profile

    Wednesday, May 4, 2016 12:07 AM

All replies

  • If there are no events logged then I do not see another way to figure out happened. In similar scenarios, I apply what is mentioned in the Wiki I started here: http://social.technet.microsoft.com/wiki/contents/articles/24170.how-to-provide-temporary-and-secure-administrative-access-to-critical-systems-and-applications.aspx

    It requires Orchestrator and it allows me to keep an eye on what people are doing. In addition, admins have no admin rights by default, they need to request for them through the self service portal. This may inspire you to have a similar control in place.


    This posting is provided AS IS with no warranties or guarantees , and confers no rights.

    Ahmed MALEK

    My Website Link

    My Linkedin Profile

    My MVP Profile

    Wednesday, May 4, 2016 12:07 AM
  • Hi Harconii,

    Have you got this issue regularly or just once?According you description,it seems like it was man-made.I assume that somebody create a A record with the IP belongs to DNS server,and delete it when he found it was wrong way.But the record is still in the cache,and when you restarted the DNS server,cache was cleared.So there is nothing has been logged.

    ________________________________________
    Best Regards,
    Cartman
    Please remember to mark the replies as answers if they help and unmark them if they provide no help. If you have feedback for TechNet Support, contact tnmff@microsoft.com.

    Wednesday, May 4, 2016 2:12 AM