none
Long execution times for Search pages even after 4th or 5th user has accessed the same page.

    Question

  • Hi Guys

    We have a Sharepoint 2010 farm with two NLB web fronts ends, a Index server and a two box SQL cluster. The same problem also occurs on a dev box which has Sharepoint 2010 and SQl server running on the same single box.

    When a user access the search or advanced search page we get long execution times of between 15 and 30 seconds.  The strange thing is that the user will have a slow response the first time, then they are fine for a period of an hour or two and then they hit the slow response again.  Below i have copied info from the developer dashboard.  Does anyone have a suggestion of where to start tackiling this issue.

    Thanks

    1st Run.  (Note this site was accessed by 3 other users first)

    • BeginRequestHandler (0.05 ms)
    • PostAuthenticateRequestHandler (0.06 ms)
    • PostResolveRequestCacheHandler (18.48 ms)
      • GetWebPartPageContent (16.87 ms)
        • GetFileAndMetaInfo (15.57 ms)
    • GetWebPartPageContent (35.25 ms)
      • GetFileAndMetaInfo (34.99 ms)
    • GetWebPartPageContent#1 (89.27 ms)
      • GetFileAndMetaInfo (89.06 ms)
    • Add WebParts (3975.17 ms)
      • Search Box (3974.54 ms)
    • SearchBoxEx.OnLoad (15426.56 ms)
      • SearchBoxEx.HandleContextualScoping (0.01 ms)
      • UserPreference.GetUserPreference (15418.52 ms)
        • UserPreference.GetFromCache (0.04 ms)
        • SearchServiceApplicationProxy.GetUserPreferenceSerializeHelper (15418.12 ms)
          • CreateChannelActingAsLoggedOnUser:Microsoft.Office.Server.Search.Administration.ISearchServiceApplication (15398.84 ms)
            • GetServiceSecurityToken (15398.58 ms)
              • CacheHandle (15398.31 ms)
                • CreateChannelActingAsLoggedOnUser:Microsoft.SharePoint.Administration.Claims.ISPWindowsTokenCacheServiceContract (15059.44 ms)
                  • GetServiceSecurityToken (15048.41 ms)
                  • CreateChannelWithIssuedToken (10.86 ms)
                • InitializeWcfOperation (0.02 ms)
                • ExecuteWcfOperation:http://tempuri.org/ISPWindowsTokenCacheServiceContract/CacheHandle (301.52 ms)
                  • CleanUpWcfOperation (0.01 ms)
            • CreateChannelWithIssuedToken (0.13 ms)
          • GetUserPreferenceSerializeHelper (18.92 ms)
            • InitializeWcfOperation (0.02 ms)
            • ExecuteWcfOperation:http://tempuri.org/ISearchSiteAdministrationServiceApplication/GetUserPreferenceSerializeHelper (17.79 ms)
              • CleanUpWcfOperation (0.01 ms)
        • UserPreference.SetToCache (0.08 ms)
    • ToolBarMenuButton.CreateChildControls for SiteActions (10.23 ms)
    • EnsureListItemsData (11.55 ms)
    • EnsureListItemsData#1 (11.08 ms)
    • ToolBarMenuButton.CreateChildControls for PersonalActions (0.14 ms)
    • SPPageStateControl:OnLoad (9.88 ms)
    • Activate web part connections (0.07 ms)
    • CachedList:Quick Deploy Items (6.86 ms)
    • SPPageStateControl:OnPreRender (247.59 ms)
      • Load CachedList:2ffb0acc-2f43-4bc0-945a-89a99f955e3b (28.10 ms)
        • CachedList:Pages (5.73 ms)
      • EnsureListItemsData (9.46 ms)
      • EnsureListItemsData#1 (20.76 ms)
    • Render Ribbon. (1.88 ms)
    • CachedArea: Caching SPQuery against against list: Tabs in Search Pages, web: /SEARCH (126.14 ms)
      • EnsureListItemsData (91.72 ms)
      • EnsureListItemsData#1 (14.04 ms)
    • Render WebPart Zone TopZone (111.60 ms)

    2nd Run

    • BeginRequestHandler (0.05 ms)
    • PostAuthenticateRequestHandler (0.07 ms)
    • PostResolveRequestCacheHandler (29.50 ms)
      • GetWebPartPageContent (27.28 ms)
        • GetFileAndMetaInfo (26.72 ms)
    • GetWebPartPageContent (31.69 ms)
      • GetFileAndMetaInfo (31.11 ms)
    • GetWebPartPageContent#1 (15.90 ms)
      • GetFileAndMetaInfo (15.28 ms)
    • Add WebParts (2.43 ms)
      • Search Box (0.35 ms)
    • SearchBoxEx.OnLoad (0.54 ms)
      • SearchBoxEx.HandleContextualScoping (0.00 ms)
      • UserPreference.GetUserPreference (0.21 ms)
        • UserPreference.GetFromCache (0.07 ms)
    • ToolBarMenuButton.CreateChildControls for SiteActions (0.33 ms)
    • EnsureListItemsData (20.14 ms)
    • EnsureListItemsData#1 (13.84 ms)
    • ToolBarMenuButton.CreateChildControls for PersonalActions (0.14 ms)
    • SPPageStateControl:OnLoad (0.15 ms)
    • Activate web part connections (0.02 ms)
    • SPPageStateControl:OnPreRender (0.55 ms)
    • Render Ribbon. (1.87 ms)
    • Render WebPart Zone TopZone (0.97 ms)
      • Render WebPart Search Box (0.87 ms)
    Friday, August 20, 2010 12:48 AM

Answers

  • From the data I see posted, the call to GetServiceSecurityToken took 15 seconds on the "slow" load, and didn't happen at all on the fast load, where it was probably cached. I expect this token times out after a while.

    This doesn't appear to be a search problem, but is something impacting the performance of you Security Token Service. If you're using Windows Authentication, this may mean the domain controller is responding slowly, which might be revealed by looking a network trace for traffic from the web front end to the DC.

     

    The only long delay that I see is that loading the Search Box webpart took almost 4 seconds on the slow load, but was loaded very fast the next time. I would be looking at SQL performance to try and improve in that area.

    Thanks,

     Manas

     

    • Marked as answer by Lily Wu Thursday, September 02, 2010 8:05 AM
    Tuesday, August 31, 2010 7:18 PM

All replies

  • From the data I see posted, the call to GetServiceSecurityToken took 15 seconds on the "slow" load, and didn't happen at all on the fast load, where it was probably cached. I expect this token times out after a while.

    This doesn't appear to be a search problem, but is something impacting the performance of you Security Token Service. If you're using Windows Authentication, this may mean the domain controller is responding slowly, which might be revealed by looking a network trace for traffic from the web front end to the DC.

     

    The only long delay that I see is that loading the Search Box webpart took almost 4 seconds on the slow load, but was loaded very fast the next time. I would be looking at SQL performance to try and improve in that area.

    Thanks,

     Manas

     

    • Marked as answer by Lily Wu Thursday, September 02, 2010 8:05 AM
    Tuesday, August 31, 2010 7:18 PM
  • Hi Kallanxli, we're having the same problems with our setup. We're working on the theory that it may have something to do with Claims Authentication (and possibly the way it works with an AD model that's earlier than 2008 Native)... as the same solution with the same content but a Classic authentication solution does not have the same problem.

    Kallanxli, are you using Claims Authentication as well? If so, what is your domain "Level"?


    Brad Saide
    brad.saide[replace with @]gmail.com
    http://kb4sp.wordpress.com
    Tuesday, May 17, 2011 1:14 AM
  • We were having this same issue, though with a different web part. The GetServiceSecurityToken frequently took 15 seconds. This KB article had the fix for us: http://support.microsoft.com/kb/2625048 .

    SharePoint 2010 uses SSL to encrypt intra-farm communication. It should work invisibly, but under the right conditions, it can cause problems, resulting in these 15 second timeouts. The fix is to export SharePoint's SSL cert through PowerShell, then add it to the server's Trusted Root Authentication Provider store through the MMC's Certificate snap-in.

    Monday, March 12, 2012 5:53 PM