none
Slow loading of shared DLL files

    Question

  • We have built a system with a Windows Server Standard 2012 as the host and two Windows 7 as the clients connected on a gigabit TCP/IP network.

    On the host, a 64-bit MFC-based executable (app) and some small-size dependent DLL files are located and shared to Everyone with the full control right. The app can be run either on the host or on the client.

    When the app is run the first time on any of the clients, it starts up quickly/normally  - Taking ~1 second - with all the DLL files being loaded quickly.

    However, the next run of the app, either on the host or on the other client, takes 3~4 min to start up. Process Explorer showed that each DLL file loading took 30~40 seconds.

    It was also found that during the slow start up, if the first client is shut down, both the ongoing and the rest of the DLL loading finished quickly giving a sudden start up of the app.

    BTW, the slow start up has never been seen on an old system with Server 2003 (host) and XP (clients).

    Anyone with similar experience?
    What would be the possible causes?
    How to fix it?

    Thanks a lot in advance.

    Jeff

    Friday, December 07, 2012 8:57 AM

Answers

  • My guess is that the first client obtains an oplock (now in SMB2/3 terms more properly called a lease) on the file(s). When the second client starts, it waits for the lease to be broken 

    Best way of verifying is taking a WireShark trace - start it before the first client runs the app, end it when the second client has succesfully loaded the dll 

    Dilip Naik

    Storage MVP

    www.winntfs.com 


    www.winntfs.com

    • Marked as answer by J.F Fan Monday, December 24, 2012 5:06 PM
    Monday, December 10, 2012 9:56 PM

All replies

  • More info:

    The slow loading of the shared DLL's happens only if the second run of the app is attempted withing ~3 min of the first run. It doesn't happen with the third and later runs.
    Also, both the app and DLL's are debug version built from VS2010 projects.

    Monday, December 10, 2012 4:30 PM
  • My guess is that the first client obtains an oplock (now in SMB2/3 terms more properly called a lease) on the file(s). When the second client starts, it waits for the lease to be broken 

    Best way of verifying is taking a WireShark trace - start it before the first client runs the app, end it when the second client has succesfully loaded the dll 

    Dilip Naik

    Storage MVP

    www.winntfs.com 


    www.winntfs.com

    • Marked as answer by J.F Fan Monday, December 24, 2012 5:06 PM
    Monday, December 10, 2012 9:56 PM
  • Hi Dilip,

    Thank you very much for the information.
    Based on it and KB296264, the issue appears to have been resolved by adding the following registry key on the server:

    Key: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\LanmanServer\Parameters\EnableOplocks Value: 0

    Again, thank you very much!

    Jeff

    Monday, December 10, 2012 10:58 PM
  • Unfortunately, the key worked only once or at most 2 times.
    "
    The opportunistic locking registry keys are valid only for traditional SMB (SMB1). You cannot turn off opportunistic locking for SMB2. SMB2 was introduced in Windows Vista to enable faster communication between computer that are running Windows Vista and Windows Server 2008 or Windows Server 2008 R2." is stated in KB296264 after reading it more carefully again.

    So, what should be done to get rid of the locking on Windows Server 2012 (SMB3) and Windows 7 (SMB2.1)? Any other possibility causing the slow loading?

    Thanks.
    Jeff

    Tuesday, December 11, 2012 1:38 AM
  • Once you have a wireshark trace, I am happy to look at it and send it to correct person at Microsoft, assuming it has the relevant data needed 

    www.winntfs.com

    Tuesday, December 11, 2012 3:50 AM
  • Hi Dilip again,

    Sorry for having not done the wireshark trace, yet. After we disabled SMB2 (by adding "HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\LanmanServer\Parameters\SMB2" and setting the value to "0") in addition to disabling OpLocks on the server, the slow loading has completely gone.

    One thing we don't understand is that why no slow loading was seen with the old system based on Server 2003 and XP where the same lock was enabled by default.

    We have submitted a support case with Microsoft Support, and hopefully we can get more details soon.

    We appreciate your help very much!

    Jeff

    • Marked as answer by J.F Fan Monday, December 24, 2012 5:03 PM
    • Unmarked as answer by J.F Fan Monday, December 24, 2012 5:05 PM
    Tuesday, December 11, 2012 7:09 AM
  • Hi,

    It seems the oplocking is the cause, so have you tried the values provided in KB296264 for tuning the performance?

    The following value specifies the minimum link throughput that the server allows before it disables raw and opportunistic locks for this connection:

    MinLinkThroughput
    REG_DWORD 0 to infinite bytes per second
    Default: 0

    The following value specifies the maximum time that is allowed for a link delay. If delays exceed this number, the server disables raw I/O and opportunistic locking for this connection.

    MaxLinkDelay
    REG_DWORD 0 to 100,000 seconds
    Default: 60

    The following value specifies the time that the server waits for a client to respond to an oplock break request. Smaller values allow detection of crashed clients more quickly, but might potentially cause loss of cached data.

    OplockBreakWait
    REG_DWORD 10 to 180 seconds
    Default: 35


    TechNet Subscriber Support in forum |If you have any feedback on our support, please contact tnmff@microsoft.com.

    Tuesday, December 11, 2012 7:19 AM
  • Hi Shaon,

    Thanks for the information.
    We only tried OplockBreakWait and by setting it to the shortest 10 sec the loading did become faster but it was still not acceptable.

    Jeff

    Wednesday, December 12, 2012 5:20 AM
  • Hi Jeff,

    So it seems that the app you are using works improperly under SMB2. Is it acceptable to disable SMB2 and using SMB1 as a workaround? And as you have already submit a support case, let's see if there is a solution rather than the workaround.


    TechNet Subscriber Support in forum |If you have any feedback on our support, please contact tnmff@microsoft.com.

    Thursday, December 13, 2012 1:59 AM
  • Hi Shaon,
    Sorry for the late response.
    Yes. The performance appears OK so far after disabling SMB2.
    Based on the data collected with Wireshark, the client-side (Win7/WES7) somehow failed to respond to the oplock break request from the server for each of the DLL files, causing timeout of the dialogue corresponding to the setting of  OplockBreakWait. BTW, the server didn't send any such request with SMB2 disabled.
    The support team is working on the issue and hopefully a solution is available in near future.
    Jeff

    • Marked as answer by J.F Fan Monday, December 24, 2012 5:05 PM
    • Unmarked as answer by J.F Fan Monday, December 24, 2012 5:06 PM
    Thursday, December 20, 2012 7:29 AM
  • Hello Jeff,

    We are facing the exact same problem. We migrated to a Windows Server 2012 File Server from Windows Server 2008 R2.

    Since the migration, we are facing some sort of "hick ups", and especially long waiting times for opening files which are already openened by someone other. This can take up to 60 seconds.

    I already changed the OpLockBreackWait to 10. This made very littkle effect.

    Yesterday we disabled SMB2, which did a major improvement. But this is not a situation we want...

    Is the problem resolved for you? And can you share the solution (if there is one)?

    Thank You in Advance,

    Dave

    Friday, November 22, 2013 8:10 AM