none
Sysmon on windows server causing delays to open Office files RRS feed

  • Question

  • Hi,

    We having strange issue with Sysmon. For our SIEM project we loaded Sysmon v11 on bunch of our Windows 2012r2/2016 servers with config file provided by SIEM platform. Immediately it introduced ~45 second delay whenever you open office files (e.g Exel’s xlsx) from workstation (win10) on all file servers. No matter what workstation, physical or virtual server, win2012 or win2016, office 365 or office 2010. It’s persistent even on freshly installed Server 2016 with no additional software installed, so it’s definitely not effect of other software. This happens only when you open file with Office application from network file share, copying file or opening it with other app is fine. Example – open file with excel from mapped drive, Excel starts and logo shows – Downloading: … (0%) for 45 sec.

    In default Sysmon config this problem does not exist. Playing with provided config file we’ve narrowed it down to these 2 sections: FileCreate and FileCreateStreamHash. If we remove these events from config file problem goes away. Include/exclude don’t matter. Tried few other config files like github /SwiftOnSecurity project, same deal.

    Any suggestions would we quite appreciated. Thanks!

    Friday, May 22, 2020 9:00 PM

All replies

  • I have added this to the Sysmon backlog and will try to reproduce it. It would help us expedite this though if you were able to provide us with a Process Monitor log for the duration of the open. If you are willing and able to assist us with this could you contact me offline at syssite@microsoft.com and I will provide you with an upload location.

    MarkC(MSFT)

    Tuesday, May 26, 2020 6:08 AM
  • MarkC, i've emailed you.

    Meantime, this is config file causing problem above.

    cybersecurity.att.com/documentation/resources/downloads/sysmon_config_schema4_0.xml

    Thanks.

    Tuesday, May 26, 2020 4:06 PM
  • Thanks for the procmon logs. I can see from those that there is  a 35 second delay when Sysmon attempts to process an open request for the .xlsx file. We are currently trying to trace the root cause of this and will get back to you shortly.

    MarkC(MSFT)

    Friday, May 29, 2020 10:06 AM
  • We've rolled back to previous version 10.42 and the problem solved, no delay anymore. Hopefully this will help to pinpoint a problem, definitely showed up in v11. Is 10.42 stable enough to be used while waiting for an update? Thanks.

    Tuesday, June 2, 2020 3:48 PM
  • Hello

    sorry for the delay in responding. I was able to see the 35 second delay in the procmon log you sent me. I am currently finalising a fix for another Sysmon issue but this is next on the list and hoping to get to it tomorrow.

    MarkC(MSFT)

    Tuesday, June 2, 2020 4:41 PM
  • Hello Mark,

    Is there any progress on looking into this issue?  Thanks.

    Monday, June 15, 2020 2:09 PM
  • Hi

    We have the same issue as well see. Can we pleas get access to any fix

    Tuesday, June 16, 2020 7:10 AM
  • Hi Just to add we had the same issue but when we removed the File creation and alternate data streams section the issue did resolve. If we can get access to the fix that would be great

    Andy

    Tuesday, June 16, 2020 7:12 AM
  • Hi Mark,

    we also have the same issues.

    And we found that some tmp files will appear at folders which the excel file open and saved.

    Besides, the excel will have a popup and said that sharing conflict.

    Hope you have updated for it.

    Friday, June 19, 2020 7:08 AM
  • Hi all

    sorry for the delay. This has been resolved for Sysmon 11.10 which was supposed to be published yesterday. I will prod Mark R. again to see when we can get this pushed out but in the meantime if anybody would like a pre-release version ahead of publishing contact me offline at syssite@microsoft.com and I will make this available.

    MarkC(MSFT)

    Friday, June 19, 2020 7:48 AM
  • Hi Mark,

    I've confirmed I'm pulling down v11.10 now from your Sysmon.zip file host.

    PS C:\WINDOWS\system32> Get-FileHash 'C:\Users\ZF10WIN10\Documents\Sysmon\Sysmon.exe' | Format-List
    
    
    Algorithm : SHA256
    Hash      : CB83E1C439BE03F7DA29EDFB4452FD5EE21FA616878BD70F4D44F77355704C3E
    Path      : C:\Users\ZF10WIN10\Documents\Sysmon\Sysmon.exe
    
    
    
    PS C:\WINDOWS\system32> (Get-Item C:\Users\ZF10WIN10\Documents\Sysmon\Sysmon.exe).VersionInfo.FileVersion
    11.10

    Can you confirm that version has the fix for the issue raised in this post?

    Thanks,

    Chris

    Wednesday, June 24, 2020 7:20 PM
  • Hi Chris

    yes this was resolved in 11.10

    MarkC(MSFT)

    Thursday, June 25, 2020 7:29 AM
  • Thanks Mark!
    Thursday, June 25, 2020 2:00 PM
  • I can confirm that original problem in this topic is resolved with v11.10. Tested on all affected systems.

    Thank you for a fix.

    Thursday, June 25, 2020 3:59 PM
  • Hi Chris

    thanks for confirming. Much appreciated.

    MarkC(MSFT)

    Friday, June 26, 2020 9:58 AM
  • Mark, 

    I don't want to start a new topic, it would require proofs and such.

    I just want to mention it here so you guys can keep an eye on it.

    It seems that v11.10 has possibly introduced some memory leak, at least using ATT configuration file mentioned in my 3d reply.

    On many of our systems new Sysmon process slowly grew in memory consumption. For example one system which had 11.10 for a week was using ~1 gb of ram when checked. I restarted one of our servers - next day it showed ~250 mb used, while previous versions were always <10 mb.

    Again, i don't want to point at the new version, but you can probably easily check.

    Thanks!


    Thursday, July 2, 2020 1:18 AM
  • Hi Andrei

    Would it be possible to share a process memory dump of the sysmon process? (If you never created one before you can simply right click on sysmon.exe in task manager and select create dump file from the context menu).

    If you are happy to share the dump could you contact me offline at syssite@microsoft.com and I will arrange to collect it.

    MarkC(MSFT)

    Thursday, July 2, 2020 7:51 AM