none
IAS crashs with msvcrt.dll under WIndows Server 2008 32bits

    Question

  • Hi,

    I have two servers with Windows Server 2008 32 bits which fails and becomes unresponsive.

    The event log says:

    "Faulting application svchost.exe_IAS, version 6.0.6001.18000, time stamp 0x47918b89, faulting module msvcrt.dll, version 7.0.6002.18005, time stamp 0x49e0379e, exception code 0xc0000005, fault offset 0x0000a1c3, process id 0x414, application start time 0x01c9e9d991aa7614."

    The servers are fresh installed with the lastest patches installed and only with Firefox, Snort and WireShark applications installed.

    The roles installed are only NAP role for one of them and the second have NAP, IIS and Certification Authority. The are both Global Catalog and Domain Servers. The system is unable to recover and I have to push the reset button to recover :-s

    I have an third 2008 server 64 bits, with NAP and Routing Service roles installed, and  it do not fails (it is PDC, RID Master, Schema Master etc...., and not GC)

    Thanks in advance for any ideas.

    H.
    Friday, June 12, 2009 10:23 PM

Answers

  • HI all,
       Finally the train as arrived. I request everyone here to update theirs windows with the latest security update via Windows Update.
       Kindly visit http://windowsupdate.microsoft.com and get your machines fully patched.

     Thanks a lot for everyone for patiently waiting for the fix.

    Thanks
    -RamaSubbu SK
    Sorry! Microsoft doesn't own any liability & responsibility for any of my posting.
    Tuesday, December 08, 2009 9:54 PM

All replies

  • Hi 
     It looks like some 3rd party application overrites the msvcrt.dll to the old version. Can you uninstall all the 3rd party applications (Firefox, Snort, WireShark and whatever you have installed) and see whether it repros or not .

    Thanks
    -RamaSubbu SK
    Sorry! Microsoft doesn't own any liability & responsibility for any of my posting.
    Saturday, June 13, 2009 11:20 PM
  • Hi!

    Thanks for your reply.

    I uninstall all the applications, but the msvcrt.dll still having the same version number, and we still having the same problem :-s

    Now, I make a set of tests and be able to reproduce the crash:

    When a User tries to login to our Wireless network and negotiaites with the radius server using PEAP protocol and a wrong password, generates the failure.

    I use a Mac OS Leopard and IPhone clients to reproduce the problem.

    Can I ask you a favor? If you have access to a Server 2008 32 bits, send me the msvcrt.dll's SHA1 or MD5 checksums to verify my libraries?

    Thanks in advance.

    H.

     
    Monday, June 15, 2009 9:08 AM
  • Thanks for reply with more information. With your reproduce steps that it is clear that is not because of 3rd party application. You can re-install those applications again, if needed. Sorry to say we need 2 more information/help from you 1. IAS Tracing/log and 2. Crash Dump of IAS Service. This doesn't look like MSVCRT.dll issue.

    If you can do this , I will send you the steps to do this.


    Thanks
    -RamaSubbu SK
    Sorry! Microsoft doesn't own any liability & responsibility for any of my posting.
    Monday, June 15, 2009 6:22 PM
  • Hi RamaSubbu,

    Yes, I am able to get all this information. I am just running the IAS service in one of the servers and can make tests over them without several damages in my organization.

    Just, I will need help about how to get all the information you need.

    Thanks in advance.

    H.
    Monday, June 15, 2009 6:45 PM
  • Thanks a lot for ready to send us the informations we needed.
    From the above in the thread, I assume that you have 32-Bit OS. Kindly do the following.

    (1) Install the Windows Debugger fro http://msdl.microsoft.com/download/symbols/debuggers/dbg_x86_6.11.1.404.msi
          Default Installation Location: C:\Program Files\Debugging Tools for Windows\
         You can customize it if you want.

    (2) From the Elevated Command Prompt, Execute the following commands, 
          (A) netsh ras set tr * en     
                        This will enable all the tracing in the IAS components.
     
          (B) "C:\Program Files\Debugging Tools for Windows\windbg.exe" -psn IAS -g -n
                    Select "yes" it is asking to save the workspace or anything else.
                     This launch the WIndows debugger and attach to the IAS Server Process.
    (3) Now reproduce the issue by try authenticating with wrong password (make it crash).
    (4)  Now if you check the WIndows Debugger, there is will prompt "0:0?>", in the prompt type .Dump /f c:\IASCrash.dmp
           This will take crash dump with full process memory of the current state.
     (5) Once the memory dump is successfully written, type .detach in the debugger and then you can close the Windows Debugger application.
    (6)Now again from the elevate command prompt to disable the enabled tracing in the 2nd step, execute the following command.
          netsh ras set tr * di
    (7) now compress all the file C:\IASCrash.dmp and all the tracing files at %windir%/tracing, all in to a single zip file.
     (8) Upload all those into SkyDrive or any other similar service and give us the link.

    If you couldn't understand any of the steps feel free to contact us.

    Thanks
    -RamaSubbu SK

              
     


    Sorry! Microsoft doesn't own any liability & responsibility for any of my posting.
    Monday, June 15, 2009 10:51 PM
  • Hi,

    I post two files in http://cid-78651358912cbfa4.skydrive.live.com/browse.aspx/.Public

    IASDump_x86.zip which contains all your required information form a X86_32 bits server
    IASDump_x64.zip which contains all your required information from an AMD_64 bits server.

    I wrote before the IAS under Windows 2008 64 bits works well, but I was testing against this server, and fails with the same error message :-s. This server has the same applications (32 bits versions) like the 32 bits server, and Routing and Access services for NAP

    Thanks in advance.

    H.




    • Edited by hmolina Tuesday, June 16, 2009 5:21 PM Write Error
    Tuesday, June 16, 2009 1:40 PM
  • Thanks a lot for the information you sent. Thanks a lot again.
     Our experts are looking into this now. At the initial stage it looks like people are already know about this issue and working on it. But I will confirm once I got the complete confirmation from the experts.

    Thanks
    -RamaSubbu SK
    Sorry! Microsoft doesn't own any liability & responsibility for any of my posting.
    Tuesday, June 16, 2009 9:52 PM
  • Has there been any update to this?  We are currently experiencing the same issue.

    Thanks,
    -Brent
    Monday, July 13, 2009 7:52 PM
  • I am having the same issue, however, I am still having the issue even after removing the role.  It is also worth noting that I am having this issue on two seperate 2008 32 bit servers.  A solution to this issue would be great. 
    Wednesday, July 15, 2009 5:01 PM
  • Hi, after you message one month ago, I do not have any news about this issue.

    You know if it was solved in the last set of patches released yesterday?

    I will make a new test, but I preferred any news in this forum.

    Thanks in advance

    H. Molina.
    Wednesday, July 15, 2009 5:18 PM
  • After removing the NPS role, you should not face this problem. It could be someother issue. Kindly verify the IAS service is not running or not present after removing the role.

    THanks
    -RamaSubbu SK
    Sorry! Microsoft doesn't own any liability & responsibility for any of my posting.
    Friday, July 17, 2009 4:14 AM
  • Hi H. Molina,
      Sorry, I'm in a month long vacation, thats why I couldn't followup with the team. I'm still on vacation. I will try to information from the team and update you at the earliest.

    Thanks
    -RamaSubbu SK
    Sorry! Microsoft doesn't own any liability & responsibility for any of my posting.
    Friday, July 17, 2009 4:16 AM
  • Hi! I'm seeing this error too on 2008 x64 SP2. What I've observed is that when a user fails to authenticate using CHAP the event will occur, WMI will reset and brings down the following services...

    Application experience
    Ike and authip ipsec keying modules
    Ip helper
    Secondary logon
    Server
    System event notification service
    Task scheduler
    User profile service
    Background intelligent transfer service
    Network Policy Server

    This doesn't seem to occur when using EAP or PEAP and authentication fails.

    The device in question was an Apple iPhone.

    Thanks!
    Thursday, August 13, 2009 5:25 PM
  • We are experiencing the same issue on three different DCs, all running Windows Server 2008 SP2 on both x64 and x86.  All are running NPS for wireless authentication purposes.  The same services are knocked out.  The same error in the event log.  No third party apps installed.  Anyone have a solution to this yet?
    Friday, August 21, 2009 2:11 PM
  • Currently working with Premiere. Suggestions given to mitigate the total collapse are...

    1.       split 4 service (ias, rasman , lanmanserver and eaphost )to different instances of svchosts.exe so that other services do not crash.

    2.       The command that you need to run is “c:\> sc config ias type= own”. Repeat the same command by replacing the service name and then restart the server.

    Friday, August 21, 2009 8:48 PM
  • Hi,

    is there any update or fix for this as yet from premier support?? I am also experiencing this on all DCs that have NPS role installed for wireless auth.

    I have not yet split the 4 services into their own instances, hoping there might be a fix prior to doing this.

    Thanks
    Wednesday, September 02, 2009 10:22 PM
  • Hi! Just heard from premier. Fix will be added in December updates as an ETA. Until then the recommendation is to not use SP2 for full functionality. To stabilize, separate out services, but CHAP authentication will fail from my observations.
    Thursday, September 03, 2009 6:00 PM
  • I have this same problem on X64. I don't know if I can wait until December!
    Wednesday, September 16, 2009 3:21 PM
  • I Have the same problem. It's easy to reproduce. Just try to authenticate a valid user with a wrong password in wifi and boom...
    All services, as described above, crash. The only error I don't get is the IAS error. I do get the EAP one... 
    I'm using W2K8 std x64

    Is there already a hotfix available? We also use the NPS service for wifi authentication and december is a long time...
     
    Friday, October 16, 2009 2:58 PM
  • Hi, X-iT

    The final fix will be coming soon. 

    Thanks
    Qunshu

    Sorry. My posting is my personal suggestion, Microsoft won't take any responsibilities for my posting. But I am more than happy to try my best to help you.
    Saturday, October 17, 2009 9:11 PM
  • Hi,

    We are having the same problem on our 2008 x64 DC's running NPS.  I installed SP2 and this exact problem started happening.  I uninstalled SP2 and the problem went away...  A fix would be much appreciated.

    Thanks!
    Monday, October 19, 2009 4:35 PM
  • Hi,

    Could you advise if Server 2008 R2 is affected by this?

    Thanks,

    Nick
    Saturday, October 24, 2009 7:58 PM
  • Server 2008 R2 is not affected by this.
    Thanks to all for being patient and waiting for the fix for Windows Server 2008. 

    Thanks
    -RamaSubbu SK 
    Sorry! Microsoft doesn't own any liability & responsibility for any of my posting.
    Sunday, October 25, 2009 2:37 AM
  • Please post to the forum when available.
    Wednesday, October 28, 2009 5:09 PM
  • Sure, Definitely I will post once the fix is available.

    -RamaSubbu SK
    Sorry! Microsoft doesn't own any liability & responsibility for any of my posting.
    Wednesday, October 28, 2009 7:05 PM
  • Sure, Definitely I will post once the fix is available.

    -RamaSubbu SK
    Sorry! Microsoft doesn't own any liability & responsibility for any of my posting.

    Posting to add "Alert me" for this thread...
    Anthony Sheehy - MCP, MCITP
    Tuesday, November 03, 2009 1:12 PM
  • RamaSubbu: Is there already a KB number on this issue? (without the actual fix)
     
    Any updates you know of? 

    Thanks!
    Wednesday, November 11, 2009 3:49 PM
  • Sorry X-iT,
       I'm sorry to say that there is nothing as such. Thanks a lot to all for patiently waiting for the fix.

    Thanks
    -RamaSubbu SK
    Sorry! Microsoft doesn't own any liability & responsibility for any of my posting.
    Wednesday, November 11, 2009 4:06 PM
  • don't think we'll be impacted by the problem much longer...we're looking into Cisco's radius solution :)
    Wednesday, November 11, 2009 4:36 PM
  • Server 2008 R2 is not affected by this.
    Thanks to all for being patient and waiting for the fix for Windows Server 2008. 

    Thanks
    -RamaSubbu SK 

    Where did you get this info from ?
    We have a supportcase open for this issue and asked if R2 would fix the issue :

    I also finally received feedback internally if this bug is fixed in Win 2008 R2. Unfortunately this is not the case.

    And latest status for the fix :

    The fix will be included in Service Pack 3.
    I have no release date for service pack 3 to give you.
    Also I tried to get information if a KB hot fix will be released before the release of service pack 3 but was not able to extract any such information.

     ( I was posting updates on this issue to : http://social.technet.microsoft.com/Forums/en-US/winservergen/thread/173d6498-1797-4513-a173-10f7b43220f0 )

    Friday, November 13, 2009 8:54 PM
  • Having experienced this bug under Server 2008's NPS, it really is a show stopper. It makes NPS not fit for purpose in many scenarios.

    Is there any way we can get more accurate information? And, collectively, add weight to get a fix expedited and made available as a Hotfix!?

    When deployed on a domain controller for performance or rationalisation reasons, its effects are catastrophic.
    Monday, November 16, 2009 8:05 PM
  • Well, I've choked on the MS blue pill long enough...my management has stuffed the red pill down my throat and it's time to face reality...

    They want this problem fixed at a remote location and aren't satisfied with the workarounds we've put into place. They aren't buying the whole "It's a real problem but there's no KB article" story anymore.
    Monday, November 16, 2009 11:29 PM
  • any update to this ?

    unbelievable i just went through something similar with this gem:
    http://social.technet.microsoft.com/forums/en-US/itproxpsp/thread/29d8987a-6017-48bc-9972-dc8f8f80532c/

    its not like they dont have the resources to communicate effectively to their users and get a fix out the door - ____ the opensource projects i deal with have fixes in place within hours of an issue being reported

    Wednesday, November 25, 2009 7:15 PM
  • I will post the fix once it is available as soon as I can.


    Sorry. My posting is my personal suggestion, Microsoft won't take any responsibilities for my posting. But I am more than happy to try my best to help you.
    Friday, November 27, 2009 4:16 AM
  • We upgraded one of our domaincontrollers to R2 last week, and pointed al accesspoint to this DC.
    No crashed, so it looks like the issue is solved in R2.
    Friday, November 27, 2009 7:34 AM
  • Yes, the issue is solved in R2.
    Sorry! Microsoft doesn't own any liability & responsibility for any of my posting.
    Friday, November 27, 2009 7:36 AM
  • Ok I'll ask why our microsoft support engineer said it wasn't fixed in R2:

    I also finally received feedback internally if this bug is fixed in Win 2008 R2. Unfortunately this is not the case.

     

    Friday, November 27, 2009 7:45 AM
  • HI all,
       Finally the train as arrived. I request everyone here to update theirs windows with the latest security update via Windows Update.
       Kindly visit http://windowsupdate.microsoft.com and get your machines fully patched.

     Thanks a lot for everyone for patiently waiting for the fix.

    Thanks
    -RamaSubbu SK
    Sorry! Microsoft doesn't own any liability & responsibility for any of my posting.
    Tuesday, December 08, 2009 9:54 PM
  • Yes, for more information please refer to http://www.microsoft.com/technet/security/Bulletin/MS09-071.mspx.
    Sorry. My posting is my personal suggestion, Microsoft won't take any responsibilities for my posting. But I am more than happy to try my best to help you.
    Wednesday, December 09, 2009 12:02 AM
  • AnnoyedAdmin is no longer annoyed...okay at least not annoyed with this anymore
    Thursday, December 10, 2009 7:41 PM