locked
ForeFront BSOD - Random Reboots. RRS feed

  • Question

  • Hello All,

    My Datacenter recently set up Forefront for our environment.   I've rolled out Forefront Client security to a hand full of work stations and we are seeing random reboots / BSOD on the workstations.  

    Most of our workstations that have ForeFront installed are Dell's Optiplex 755.  Windows XP SP3, Office 2007, Office Communicator. 

    These reboots may happen once a day or multiple times a day. 

    I know if I turn off real time protection we don't have this problem.  

    I have logged a case with Microsoft but still don't have a resolution.

    0: kd> !analyze -v
    *******************************************************************************
    *                                                                             *
    *                        Bugcheck Analysis                                    *
    *                                                                             *
    *******************************************************************************

    IMPERSONATING_WORKER_THREAD (df)
    A workitem forgot to disable impersonation before it completed.
    Arguments:
    Arg1: 804e6fac, Worker Routine that caused this bugcheck.
    Arg2: 8a6db088, Parameter passed to this worker routine.
    Arg3: 8a6db088, Pointer to the Workitem.
    Arg4: 00000000

    Debugging Details:
    ------------------


    DEFAULT_BUCKET_ID:  DRIVER_FAULT

    BUGCHECK_STR:  0xDF

    PROCESS_NAME:  System

    LAST_CONTROL_TRANSFER:  from 8053883f to 804f9f43

    STACK_TEXT: 
    ba4dbd6c 8053883f 000000df 804e6fac 8a6db088 nt!KeBugCheckEx+0x1b
    ba4dbdac 805cff70 8a6db088 00000000 00000000 nt!ExpWorkerThread+0x1b1
    ba4dbddc 805460ee 8053868e 00000000 00000000 nt!PspSystemThreadStartup+0x34
    00000000 00000000 00000000 00000000 00000000 nt!KiThreadStartup+0x16


    STACK_COMMAND:  kb

    SYMBOL_NAME:  ANALYSIS_INCONCLUSIVE

    FOLLOWUP_NAME:  MachineOwner

    MODULE_NAME: Unknown_Module

    IMAGE_NAME:  Unknown_Image

    DEBUG_FLR_IMAGE_TIMESTAMP:  0

    FAILURE_BUCKET_ID:  0xDF_ANALYSIS_INCONCLUSIVE

    BUCKET_ID:  0xDF_ANALYSIS_INCONCLUSIVE

    Followup: MachineOwner
    ---------

     

    Thursday, June 18, 2009 5:56 PM

All replies

  • Whats your case#  Thanks


    CSS Security Support Engineer (FCS/MBSA/WUA/Incident Response) Check out my blog http://blogs.technet.com/kfalde
    Friday, June 19, 2009 2:04 PM
  • SRZ090507000370
    Friday, June 19, 2009 2:08 PM
  • We have ForeFront here also, and this debug dump looks very similar to some of the DF dumps we are getting.  One I just analyzed showed what I have below.  So far this has only been reported on our XP SP3 machines, no reports on Vista or the few test Win7 machines.  We have a full rollout of Forefront, so a few thousand machines, but roughly 20 reports so far.  We are going to pull Forefront from one of the frequent rebooters, to see if it stops, just to confirm.  I also have 1 full memory dump, but it only shows the fault was in the kernel memory space, not what caused the fault.

    : kd> !analyze -v
    *******************************************************************************
    *                                                                             *
    *                        Bugcheck Analysis                                    *
    *                                                                             *
    *******************************************************************************

    IMPERSONATING_WORKER_THREAD (df)
    A workitem forgot to disable impersonation before it completed.
    Arguments:
    Arg1: 804e6717, Worker Routine that caused this bugcheck.
    Arg2: 8a8ca1e8, Parameter passed to this worker routine.
    Arg3: 8a8ca1e8, Pointer to the Workitem.
    Arg4: 00000000

    Debugging Details:
    ------------------


    CUSTOMER_CRASH_COUNT:  1

    DEFAULT_BUCKET_ID:  DRIVER_FAULT

    BUGCHECK_STR:  0xDF

    LAST_CONTROL_TRANSFER:  from 8052cc1d to 8053767a

    STACK_TEXT: 
    f78dad6c 8052cc1d 000000df 804e6717 8a8ca1e8 nt!KeBugCheckEx+0x1b
    f78dadac 80575723 8a8ca1e8 00000000 00000000 nt!ExpWorkerThread+0x1b1
    f78daddc 804ec6d9 804e22f1 00000000 00000000 nt!PspSystemThreadStartup+0x34
    00000000 00000000 00000000 00000000 00000000 nt!KiThreadStartup+0x16


    STACK_COMMAND:  kb

    SYMBOL_NAME:  ANALYSIS_INCONCLUSIVE

    FOLLOWUP_NAME:  MachineOwner

    MODULE_NAME: Unknown_Module

    IMAGE_NAME:  Unknown_Image

    DEBUG_FLR_IMAGE_TIMESTAMP:  0

    FAILURE_BUCKET_ID:  0xDF_ANALYSIS_INCONCLUSIVE

    BUCKET_ID:  0xDF_ANALYSIS_INCONCLUSIVE

    Followup: MachineOwner
    ---------


    We also get this on some of the dumps:



    Use !analyze -v to get detailed debugging information.

    BugCheck DF, {804e6717, 8a8f0088, 8a8f0088, 0}

    Probably caused by : Unknown_Image ( ANALYSIS_INCONCLUSIVE )

    Followup: MachineOwner
    ---------

    Friday, June 19, 2009 8:49 PM
  • Sounds like a similar issue.   Windows 7,  Vista and Windows servers are not having this issue in our environment.

    Let me know if uninstalling Forefront stops the reboot issue for you as well.

    Thanks

    Saturday, June 20, 2009 7:38 PM
  • We seem to be experiencing the same issue with several machines.  I have uninstalled ForeFront from one machine and so far everything seems to be OK.  I will have a more definitive answer tomorrow afternoon.

    If I hadn't stumble dupon this post, I do not know what I would have done.  Thanks.

    Wednesday, June 24, 2009 9:04 PM
  • I'm glad that i'm not the only one seeing this issue.   

    We don't see this issue if we disable on scan protection.

    The recent update for forefront did not fix this issue.

    Keep me posted on your findings.


    Thanks
    Wednesday, June 24, 2009 9:56 PM
  • Wow.  I am having a very similiar problem.  My company also has forefront installed.  This issue seems to be affecting only a specific model of windows xp sp3 machines.  Its less than 5% of the total workstations. The random reboots are hitting maybe 2 times a week.  My minidumps are almost exactly the same as yours.

    How did you figure out it was a Forefront thing?  I was about to start yelling at the vendor of this particular model. 

    Is there a fix for this yet?
    Monday, June 29, 2009 5:56 PM
  • It only seems to happen on our optiplex 755 and 745.  What make / models are you having problems with?


    I started to notice this issue after our first install on a workstation.   What version of Office are you running?

     

    Monday, June 29, 2009 6:32 PM
  • Mine is with Lenovo Thinkcentre A55's.  We have several types in this family mainly 9265 and 8705.  They have intel 946GZ chipset if for some reason that might be part of the issue. 

    My first thought was this was somehow hardware or driver related.  I have replaced many of these systems' motherboards and also went the manufacturer for all the latest drivers and BIOS flash.  None of this seemed to help. 

    We have a mix of office 2003 and 2007.  I'm not sure if all the failing machines have one or the other in common but it might be all 2007.

    Any similarities? 
    Monday, June 29, 2009 7:02 PM
  • We have a case open with Microsoft right now, so I'm not sure if I can share much due to our enterprise support agreement.  There is a June public update for Forefront which we are starting testing now. (KB 971026)  We see this issue on some GX280s/GX620s, 744/755, D620 laptop, and one Tecra M7 laptop, so its not limited to 745/755.  The number of affected systems seems to be increasing for us, or reporting is increasing, and we are not seeing this on Vista.

    All I know for certain is that this is related to ntos-kernel, based on the dumps, likely a kernel device driver. 

    If we just finish upgrading to Vista (about 20% there) the problem would just go away, but at the same time this does need to be fixed.
    Monday, June 29, 2009 7:08 PM
  • Oh, on office, we have one machine that was Office 2003, then mix of 2007 SP1 and 2007 SP2. (we are staging the SP2 rollout, but we are seeing this issue on both sides of the staged rollout.)
    Monday, June 29, 2009 7:09 PM
  • I hate to be the bearer of bad news but that patch rolled out on June 19th for us.  I have already several users reporting that they had random reboots the week after (last week).
    Monday, June 29, 2009 7:32 PM
  • I don't know if this is the case with you but it seems as though most of the reports coming in say that the user is either in outlook looking through emails, or typing an email, or they had just sent an email.  Just FYI we have users that have pst files for archives as well. 

    At this point I am simply looking for some kind of exclusion rule or something I can do to just get all these PCs stable.  I'm starting to get more and more reports.  I would rather not fully turn off the real time protection though. 

    I am going to setup an exclusion for outlook.exe to see if that helps any just by some weird chance. 

    Any Microsoft people care to comment on this?
    Tuesday, June 30, 2009 3:35 PM
  • I really think its to do with Ouotlook as well now.   I uninstalled Outlook off a couple of PC's and have the users go through OWA.  These users have not rebooted yet.

    Tuesday, June 30, 2009 4:19 PM
  • It appears to be server based PST files, Outlook, Forefront, and Windows XP.  The DF is an impersonation fault, so my thinking is this:

    Forefront MP service runs as a local service.  It has to switch to impersonation mode to talk to a user-mounted PST.  (impersonate the user)  Somehow that impersonation isn't getting switch back out, and the kernel panics (crashes) with the impersonation fault.

    I don't recall where it is at, but sometime a long time ago (2003,2004) we were told that server based PSTs are not supported by Microsoft.  That may not be the case any more, but it has always been prone to problems.  (corruption, file locking issues, etc.)

    This seems to also explain why only some users are affected (out of a large number of XP/Forefront machines I have on my network, over 3000 such machines).

    I'm still working through my support case, but I believe this is the work-around until Forefront or Office or XP is fixed to properly handle the virus scanning or lack of virusscanning of PST files.  (perhaps just configure Forefront to not scan PST files ? )

    -Justin
    • Proposed as answer by Sami Saarinen Wednesday, July 22, 2009 1:17 PM
    Tuesday, June 30, 2009 4:29 PM
  • Wow... this could help me out in my push to eliminate PSTs.  Here is the article I have been shoving at people around here: 

    http://support.microsoft.com/kb/297019

    I'll give it a try and remove their pst and see what happens.

    Tuesday, June 30, 2009 4:40 PM
  • I placed OST and PST in exclusions.    So far so good.

    Wednesday, July 1, 2009 2:24 PM
  • I also put PST in exclusion and so far so good.
    Wednesday, July 1, 2009 6:21 PM
  • I don't know if this is the case with you but it seems as though most of the reports coming in say that the user is either in outlook looking through emails, or typing an email, or they had just sent an email.  Just FYI we have users that have pst files for archives as well. 
    Yep. I have this similar issue and have had reports that his crash is about to happen when we're using outlook. And with server based .pst files.

    Have to try to uncheck .pst checks from forefront and/or move those .pst:s to local disks..
    Friday, July 10, 2009 11:30 AM
  • My problems with this case are gone. It was really with Forefront and network based .pst's.

    Thank you Justin for this!

    Br,
    Sami Saarinen
    Wednesday, July 22, 2009 1:19 PM
  • Amazing, I just found this now. We have the same problem here in our environment. We are running Forefront on Dell Optiplex 755, 745 and 760, also Dell Laptops D620, D630 and E6400 running. We have been having the same issues with Xp machines, all our Vista machines are fine. I will get our Forefront team to make the changes to the .pst scanning and update you.
    Wednesday, July 29, 2009 9:49 PM
  • We just started getting this same BSOD after yesterday's FCS definition update.  We've gone ahead and added .pst and .ost to our exception list, so our fingers are crossed that the issue is handled.

    Thanks to everyone for posting about this!

    Russell Johnson
    Thursday, October 15, 2009 5:29 PM
  • We ran into this issue 2 weeks ago, just out of the blue. It seems that there is no fix for it currently other then excluding PST's from scanning completely.

    Did somebody opened a case with Microsoft on this issue?

    regards, Henno.

     

    Friday, December 10, 2010 10:45 AM