none
SharePoint 2016 Analytics - Usage reports not showing data RRS feed

  • Question

  • Hello,

    I am getting all zeros or not data whether I go to polularity and search reports or whether I go to site web analytics reports.  I have confirmed the following:

    I am showing screenshot of reports not working and results of Get-SPUsageDefinition for "Analytics" and "Page Requests".

    Any assistance or direction would be appreciated!


    Rumi

    Tuesday, July 10, 2018 12:11 AM

Answers

  • Hello:

    I'm sorry for not updating this - I'm glad I had an alert on this thread.  Yes, we solved the issue but I actually had to open a case with MS to dig into this.  Pleas follow the following steps thoroughly:

    Usage Analytics & its process

    There are various versions of “Usage Analytics” currently referred in SharePoint 2013 & 2016. Which was replaced from “Web Analytics” in SharePoint 2010 & “Usage Reporting” in Sharepoint 2007 / Moss. There are two Analytics components introduced in SharePoint 2013 which is carried over to SharePoint 2016. Search analytics and Usage analytics. Search analytics analyzes content in the search index, and usage analytics analyzes the user actions. In this post we will only discuss Usage Analytics.

    Usage Analytics Description – In simple words, Usage analytics analyzes user actions, or usage events, such as clicks or viewed items, on the SharePoint site. These Usage statistics are then combined with information about crawled content from the Search analyses, and processed. Statistics on the different usage events is added to the search index and sent to the Analytics reporting database.

    Usage Events – Every time that a user views an item, the system issues a usage event. In SharePoint 2013 three usage events are defined out of the box . These default events are always registered and analyzed by SharePoint.

    Following are the default usage events

    • Views      – number of views for a single item, page, or document.
    • Recommendations      Displayed – number of times a single item, page or document was displayed      as a recommendation.
    • Recommendation      Clicks – number of times a single item, page or document was clicked when      it was displayed as a recommendation.

    How Usage analytics works? Every time that a user views an item, the system issues a usage event and adds this to the total and recent usage event counts for that item.Statistics on the different usage events is then added to the search index and sent to the Analytics reporting database.

    How do you Use these Usage analytics – You can use the data that is generated by usage events in the following ways:

    • Show      recommendations or popular items on your site. By using recommendations,      you can guide users to other content that may be relevant for them, based      on how users previously interacted with the site. You can add      recommendations to a page by adding Recommended Items and Popular Items      Web Parts.
    • Use Content Search      Webpart (CSWP) and sort the search results by the      number of counts of a usage event.
    • View      the usage event data in the Most      Popular Items usage report. This report applies to      all items in a library, and lists the most popular items for each usage      event — for example, a list of the most viewed pages in a library.
    • View      the usage event data in the Popularity      Trends report. This report applies to a Site Collection,      a Site or an individual item in a library or list. The report shows the      daily and monthly counts for each usage event — for example, the      total views of a page on a specific day.

    What happens in the backend ?

    • You      need to first configure what data you need for analysis by enabling them      at http:// <Central Admin URL>/_admin/LogUsage.aspx.
    • The      usage files are stored at “C:\Program Files\Common Files\microsoft      shared\Web Server Extensions\16\LOGS\RequestUsage”
    • Timer      job “Microsoft SharePoint Foundation Usage Data Import” runs every five      minutes and Imports usage log files into the logging database.

    If you look in to the importProgress file it will have the latest usage file data in it with value -1 in the second line which is expected. Please find below screenshot which has the working and non-working cases. If we see the logs getting accumulated then it means the above timer job is not functioning correctly.

    • Timer      job “Microsoft SharePoint Foundation Usage Data Processing” runs daily, in      the morning around 1:00 AM and make the data ready for analysis.

    Most of the time it’s been observed that either the timer job do not run as expected or event receiver is not registered correctly. We can make use of below PowerShell to check / troubleshoot.

    # Get the service instance of "Usage and Health Data Collection Service Application"

    $c=Get-SPServiceApplication | where {$_.Typename -eq "Usage and Health Data Collection Service Application"}

    $c.ServiceInstances | fl Server, Tynename, Id, Status

    # Ran below command to check if timer job "Microsoft SharePoint Foundation Usage" is running or not

    $SIs = Get-SPServiceInstance -Identity <GUID TAKEN FROM ABOVE COMMAND>

    $Farm = Get-SPFarm

    $FarmTimers = $Farm.TimerService.Instances

    $1onLine = $false

    foreach ($SI in $SIs)

    {If($SI.Status -eq "Online")

    {$1onLine = $true

    foreach ($FT in $FarmTimers)

    {if($SI.Server.Address.ToString() -eq $FT.Server.Name.ToString())

    {if($FT.AllowServiceJobs -eq $false)

    {write-host -ForegroundColor Red "Service jobs are NOT enabled on Usage Service server" $FT.Server.Name.ToString()

    write-host -ForegroundColor green "Enabling service jobs on server: " $FT.Server.Name.ToString()

    $FT.AllowServiceJobs = $true

    $FT.Update()

    }

    else{write-host -ForegroundColor green "Service jobs already enabled on server: " $FT.Server.Name.ToString()}

    }

    }

    }

    }

    If($1onLine -eq $false ){write-host -ForegroundColor red "No instances of Usage Service online in this farm!"}

    # Ran below command to get analytics event receiver

    $ad = Get-SPUsageDefinition | where {$_.Name -like "Analytics*"}

    $vru = $ad.Receivers

    $vru

    # Ran below command to get Page Request event receiver

    $pd = Get-SPUsageDefinition | where {$_.Name -like "Page Requests"}

    $pru = $pd.Receivers

    $pru

    ##Expected receiver assemblies as below

    SharePoint 2010 : Microsoft.Office.Server.Search.Applications, Version=14.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c

    SharePoint 2013 : Microsoft.Office.Server.Search.Applications, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c

    SharePoint 2016 : Microsoft.Office.Server.Search.Applications, Version=16.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c

    # If the receiver assembly do not match the SharePoint version then we need to delete using below script

    $vru.Delete()

    $pru.Delete()

    # Ran below command to create event receivers for Analytics

    $ad = Get-SPUsageDefinition | where {$_.Name -like "Analytics*"}

    $ad.Receivers

    $ad.Receivers.Add("Microsoft.Office.Server.Search.Analytics.Internal.AnalyticsCustomRequestUsageReceiver")

    # Ran below command to verify the same

    $vru = $ad.Receivers | ?{$_.name -like "Microsoft.Office.Server.Search.Analytics.Internal.AnalyticsCustomRequestUsageReceiver"}

    $vru

    # Ran below command to create event receivers for Page request

    $pd = Get-SPUsageDefinition | where {$_.Name -like "Page Requests"}

    $pd.Receivers

    $pd.Receivers.Add("Microsoft.Office.Server.Search.Analytics.Internal.ViewRequestUsageReceiver")

    # Ran below command to verify the same

    $acru = $pd.Receivers | ?{$_.name -like "Microsoft.Office.Server.Search.Analytics.Internal.ViewRequestUsageReceiver"}

    $acru

    # Ran below command to start the analysis timer job

    $a = Get-SPTimerJob -Type Microsoft.Office.Server.Search.Analytics.AnalyticsJobDefinition

    $sa = $a.GetAnalysis("Microsoft.Office.Server.Search.Analytics.SearchAnalyticsJob")

    $sa.StartAnalysis()

    # Ran below command to get Analysis info.. Make sure to wait for 15 minutes as this will allow to complete the timer job properly

    $sa.GetAnalysisInfo()

    # Ran below command to start the usage log file import timer job

    $tj = Get-SPTimerJob -Identity ("job-usage-log-file-import")

    $tj.RunNow()

    Now, we need to let the timer job “Microsoft SharePoint Foundation Usage Data Processing” complete(This will not work if you run it manually). Once, the job completes as per its schedule we should be able to view the usage report data.


    Rumi

    • Marked as answer by Rumi's Point Wednesday, October 31, 2018 3:50 PM
    Tuesday, October 30, 2018 10:31 PM

All replies

  • Hi,

    It looks an analytic data is not being read periodically.

    Can you please check the  LOGS folder under 14/15/16 hive(s).There you must see the .usage files and temp files.

    And ensure the Event STore under C:\Program Files\Microsoft Office Servers\15.0\Data\Office Server folder with GUID. If you are not seeing those folders , probably your FARM is not configured or unhealthy.

    If you configure the LOG in different location please check or make it available those files to be read analytics timer  job.


    Murugesa Pandian MCSA,MCSE,MCPD

    Gear up for some solid action by doing.
    


    Tuesday, July 10, 2018 1:22 AM
  • Hi Rumi,

    Make sure your search service is running and items have been crawled.

    You can try the following troubleshooting steps to diagnose your issue:

    1. Make sure the "Usage and Health Data Collection" was configured.

    a. Go to Central Admin > Monitoring > Configure Usage and Health Data Collection.

    b. Validate that Enable Usage Data Collection was checked. 

    c. Validate that Events were selected to log.

    d. Validate the Log File Location exists (SAVE THIS FOR THE NEXT STEP), it's usually "c:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\16\LOGS\" .

    e. Validate the Database Server and Name of database are set and correct.

    2. Verify that the WSS_Admin_WPG and WSS_WPG groups both had full access to the Log File Location (FROM ABOVE) on all the servers.

    3. Check the WSS_Logging  (the database name configured in the usage service application configuration) database's permissions, the web service web application account and the farm account both have ownership rights over it.

    4. Make sure that Event Store Folder exists on those servers which Run the Search 2016 Analytics Component, it is generally at this location: C:\Program Files\Microsoft Office Servers\16.0\Data\Office Server.

    5. Check the contents of the WSS_Logging database, the request tables are populated with information.

    6. Make sure that table RequestUsage in WSS_UsageApplication database is populated with correct information.

    Reference(although the article is for SP2013, but also useful for SP 2016):

    http://snowburnt.blogspot.com/2014/12/how-to-troubleshoot-sharepoint-usage.html 

    If above methods do not work, you can try to delete the service application from Manage Service Applications and recreate it by going to Monitoring > Configure Usage and Health Data Collection and checking the Enable usage data collection box. Then waiting for one day and check the results.

    Best regards,

    Allen Bai


    Please remember to mark the replies as answers if they helped. If you have feedback for TechNet Subscriber Support, contact tnmff@microsoft.com.


    Click here to learn more. Visit the dedicated forum to share, explore and talk to experts about Microsoft Teams.

    Tuesday, July 10, 2018 3:04 AM
  • 1. Confirmed

    a. confirmed

    b. confirmed

    c. confirmed

    d. confirmed

    e. confirmed

    2. confirmed

    3. Farm account has "owner" web app pool account has "SPReadOnly"

    4. Confirmed

    5. tables are populated with info - I see the following table name scheme: "dbo.RequestUsage_Partition0".  It goes all the way to 31

    6. It has information but not sure what you mean by "correct"

    Please advise.

    Thank you,


    Rumi

    Tuesday, July 10, 2018 11:28 PM
  • Hi Rumi,

    What do you mean "web app pool account has "SPReadOnly"?

    Have you tried the suggestion provided by Murugesa Pandian? Have you tried the methods in the article that I mentioned in above reply?

    If you have tried all the suggestions above, for you situation, I suggest you deleted the "Health and usage service application" in "manage service applications" section in Central Admin. Then re-enabled it by going to monitoring > Configure usage and Health Data Collection and then checking the Enable usage data collection and enable health data collection check boxes and validating the other information. 

    Best regards,

    Allen 


    Please remember to mark the replies as answers if they helped. If you have feedback for TechNet Subscriber Support, contact tnmff@microsoft.com.


    Click here to learn more. Visit the dedicated forum to share, explore and talk to experts about Microsoft Teams.

    Wednesday, July 11, 2018 7:46 AM
  • in item #3, you mention to check the database permissions for the WSS_LOGGING DB.  The farm account has owner role in SQL for that database and the Web Application pool account has SPReadOnly role in SQL.  So, that's what I mean.

    Yes, I have followed the suggestions above provided by Murugesa Pandian.  Anyway, I deleted the Health and usage Service application and recreated it.  I will see if we get any analytics reports now.

    We have the Standard license.  Do you know if Usage Analytics is available to Standard license?


    Rumi

    Wednesday, July 11, 2018 11:52 AM
  • Hi Rumi,

    Yes, Usage Analytics is available to Standard license.

    Giving your Web Application pool account "Owner" permission of the database. Then compare the results.

    Best regards,

    Allen Bai


    Please remember to mark the replies as answers if they helped. If you have feedback for TechNet Subscriber Support, contact tnmff@microsoft.com.


    Click here to learn more. Visit the dedicated forum to share, explore and talk to experts about Microsoft Teams.

    Thursday, July 12, 2018 3:20 AM
  • Hi Allen,

    After recreating the Health and Usage service application and granting "Owner" permission to the database, I still do not see data in the reports.


    Rumi

    Monday, July 16, 2018 12:00 PM
  • Hi Rumi,

    Make sure your web analytics service application is started.

    Try to deactivate and Activate the site collection Features on Reporting.

    Also do IISRESET then run a full crawl and check the results.

    Check if below article is useful to you:

    https://sharepoint.stackexchange.com/questions/206372/sp2016-site-usage-logging-not-working-no-online-instance-for-service-microsoft

    Best regards,

    Allen Bai


    Please remember to mark the replies as answers if they helped. If you have feedback for TechNet Subscriber Support, contact tnmff@microsoft.com.


    Click here to learn more. Visit the dedicated forum to share, explore and talk to experts about Microsoft Teams.

    Wednesday, July 18, 2018 2:40 AM
  • Hi,

    How are things going? Have you solve your issue?

    Best regards,

    Allen Bai



    Please remember to mark the replies as answers if they helped. If you have feedback for TechNet Subscriber Support, contact tnmff@microsoft.com.


    Click here to learn more. Visit the dedicated forum to share, explore and talk to experts about Microsoft Teams.

    Tuesday, July 24, 2018 2:53 AM
  • Hi,

    were you able to fine the solution? we are facing the same issue.

    Tuesday, October 30, 2018 2:09 PM
  • Hello:

    I'm sorry for not updating this - I'm glad I had an alert on this thread.  Yes, we solved the issue but I actually had to open a case with MS to dig into this.  Pleas follow the following steps thoroughly:

    Usage Analytics & its process

    There are various versions of “Usage Analytics” currently referred in SharePoint 2013 & 2016. Which was replaced from “Web Analytics” in SharePoint 2010 & “Usage Reporting” in Sharepoint 2007 / Moss. There are two Analytics components introduced in SharePoint 2013 which is carried over to SharePoint 2016. Search analytics and Usage analytics. Search analytics analyzes content in the search index, and usage analytics analyzes the user actions. In this post we will only discuss Usage Analytics.

    Usage Analytics Description – In simple words, Usage analytics analyzes user actions, or usage events, such as clicks or viewed items, on the SharePoint site. These Usage statistics are then combined with information about crawled content from the Search analyses, and processed. Statistics on the different usage events is added to the search index and sent to the Analytics reporting database.

    Usage Events – Every time that a user views an item, the system issues a usage event. In SharePoint 2013 three usage events are defined out of the box . These default events are always registered and analyzed by SharePoint.

    Following are the default usage events

    • Views      – number of views for a single item, page, or document.
    • Recommendations      Displayed – number of times a single item, page or document was displayed      as a recommendation.
    • Recommendation      Clicks – number of times a single item, page or document was clicked when      it was displayed as a recommendation.

    How Usage analytics works? Every time that a user views an item, the system issues a usage event and adds this to the total and recent usage event counts for that item.Statistics on the different usage events is then added to the search index and sent to the Analytics reporting database.

    How do you Use these Usage analytics – You can use the data that is generated by usage events in the following ways:

    • Show      recommendations or popular items on your site. By using recommendations,      you can guide users to other content that may be relevant for them, based      on how users previously interacted with the site. You can add      recommendations to a page by adding Recommended Items and Popular Items      Web Parts.
    • Use Content Search      Webpart (CSWP) and sort the search results by the      number of counts of a usage event.
    • View      the usage event data in the Most      Popular Items usage report. This report applies to      all items in a library, and lists the most popular items for each usage      event — for example, a list of the most viewed pages in a library.
    • View      the usage event data in the Popularity      Trends report. This report applies to a Site Collection,      a Site or an individual item in a library or list. The report shows the      daily and monthly counts for each usage event — for example, the      total views of a page on a specific day.

    What happens in the backend ?

    • You      need to first configure what data you need for analysis by enabling them      at http:// <Central Admin URL>/_admin/LogUsage.aspx.
    • The      usage files are stored at “C:\Program Files\Common Files\microsoft      shared\Web Server Extensions\16\LOGS\RequestUsage”
    • Timer      job “Microsoft SharePoint Foundation Usage Data Import” runs every five      minutes and Imports usage log files into the logging database.

    If you look in to the importProgress file it will have the latest usage file data in it with value -1 in the second line which is expected. Please find below screenshot which has the working and non-working cases. If we see the logs getting accumulated then it means the above timer job is not functioning correctly.

    • Timer      job “Microsoft SharePoint Foundation Usage Data Processing” runs daily, in      the morning around 1:00 AM and make the data ready for analysis.

    Most of the time it’s been observed that either the timer job do not run as expected or event receiver is not registered correctly. We can make use of below PowerShell to check / troubleshoot.

    # Get the service instance of "Usage and Health Data Collection Service Application"

    $c=Get-SPServiceApplication | where {$_.Typename -eq "Usage and Health Data Collection Service Application"}

    $c.ServiceInstances | fl Server, Tynename, Id, Status

    # Ran below command to check if timer job "Microsoft SharePoint Foundation Usage" is running or not

    $SIs = Get-SPServiceInstance -Identity <GUID TAKEN FROM ABOVE COMMAND>

    $Farm = Get-SPFarm

    $FarmTimers = $Farm.TimerService.Instances

    $1onLine = $false

    foreach ($SI in $SIs)

    {If($SI.Status -eq "Online")

    {$1onLine = $true

    foreach ($FT in $FarmTimers)

    {if($SI.Server.Address.ToString() -eq $FT.Server.Name.ToString())

    {if($FT.AllowServiceJobs -eq $false)

    {write-host -ForegroundColor Red "Service jobs are NOT enabled on Usage Service server" $FT.Server.Name.ToString()

    write-host -ForegroundColor green "Enabling service jobs on server: " $FT.Server.Name.ToString()

    $FT.AllowServiceJobs = $true

    $FT.Update()

    }

    else{write-host -ForegroundColor green "Service jobs already enabled on server: " $FT.Server.Name.ToString()}

    }

    }

    }

    }

    If($1onLine -eq $false ){write-host -ForegroundColor red "No instances of Usage Service online in this farm!"}

    # Ran below command to get analytics event receiver

    $ad = Get-SPUsageDefinition | where {$_.Name -like "Analytics*"}

    $vru = $ad.Receivers

    $vru

    # Ran below command to get Page Request event receiver

    $pd = Get-SPUsageDefinition | where {$_.Name -like "Page Requests"}

    $pru = $pd.Receivers

    $pru

    ##Expected receiver assemblies as below

    SharePoint 2010 : Microsoft.Office.Server.Search.Applications, Version=14.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c

    SharePoint 2013 : Microsoft.Office.Server.Search.Applications, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c

    SharePoint 2016 : Microsoft.Office.Server.Search.Applications, Version=16.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c

    # If the receiver assembly do not match the SharePoint version then we need to delete using below script

    $vru.Delete()

    $pru.Delete()

    # Ran below command to create event receivers for Analytics

    $ad = Get-SPUsageDefinition | where {$_.Name -like "Analytics*"}

    $ad.Receivers

    $ad.Receivers.Add("Microsoft.Office.Server.Search.Analytics.Internal.AnalyticsCustomRequestUsageReceiver")

    # Ran below command to verify the same

    $vru = $ad.Receivers | ?{$_.name -like "Microsoft.Office.Server.Search.Analytics.Internal.AnalyticsCustomRequestUsageReceiver"}

    $vru

    # Ran below command to create event receivers for Page request

    $pd = Get-SPUsageDefinition | where {$_.Name -like "Page Requests"}

    $pd.Receivers

    $pd.Receivers.Add("Microsoft.Office.Server.Search.Analytics.Internal.ViewRequestUsageReceiver")

    # Ran below command to verify the same

    $acru = $pd.Receivers | ?{$_.name -like "Microsoft.Office.Server.Search.Analytics.Internal.ViewRequestUsageReceiver"}

    $acru

    # Ran below command to start the analysis timer job

    $a = Get-SPTimerJob -Type Microsoft.Office.Server.Search.Analytics.AnalyticsJobDefinition

    $sa = $a.GetAnalysis("Microsoft.Office.Server.Search.Analytics.SearchAnalyticsJob")

    $sa.StartAnalysis()

    # Ran below command to get Analysis info.. Make sure to wait for 15 minutes as this will allow to complete the timer job properly

    $sa.GetAnalysisInfo()

    # Ran below command to start the usage log file import timer job

    $tj = Get-SPTimerJob -Identity ("job-usage-log-file-import")

    $tj.RunNow()

    Now, we need to let the timer job “Microsoft SharePoint Foundation Usage Data Processing” complete(This will not work if you run it manually). Once, the job completes as per its schedule we should be able to view the usage report data.


    Rumi

    • Marked as answer by Rumi's Point Wednesday, October 31, 2018 3:50 PM
    Tuesday, October 30, 2018 10:31 PM
  • Hi Rumi,

    I am glad that you have solved your issue.

    You could mark your own reply as an answer. This will close this thread and other people who with similar issue will directly see your answer.

    Best regards,

    Allen Bai


    Please remember to mark the replies as answers if they helped. If you have feedback for TechNet Subscriber Support, contact tnmff@microsoft.com.


    Click here to learn more. Visit the dedicated forum to share, explore and talk to experts about Microsoft Teams.

    Wednesday, October 31, 2018 2:16 AM
  • Hi Rumi,

    I have issue with one specific SharePoint site , i am able to pull reports from all the other site collection. 

    This is in Sp 2016 env. 

    Any idea how to fix that. 

    I executed the troubleshooting steps you shared , everything is green.Still a single sharepoint site is not giving the reports.


    Thanks! Palani K

    Wednesday, November 21, 2018 2:36 PM