Using the SharePoint 2013 analytics feature

Managing analytics

Nicki Borell

by Nicki Borell on 7/14/2014

Share this:
Print

Article Details

Date Revised:
7/14/2014


Sponsored by


When you prepare for managing the SharePoint 2013 Analytics feature, you need to think about 4 main things: data collecting, files and folder locations, timer jobs, and PowerShell.

Data collection: Files and folders

As descripted in the first part in this series, data about user interactions is collected from the ULS logs from the WFE servers. Different timer jobs organize the data extraction from the ULS logs, as Figure 1 shows:

1

The SharePoint Foundation Usage Data Import timer job processes usage information into the Event Store folders. The Analytics Timer Job for Search Service Application periodically schedules analytics for Search Service Application. The search analytics component processes the data from the ULS logs.

During the installation process, you can configure the location for the Office Server. By default, the locations is %system drive%\Program Files\. This means that the folders for the Event Store and the working folders for usage analytics, search analytics and search reports are located on the system drive. There is no supported way to change this location after the installation is done. So be careful about what you’re configuring.

After the installation is done you have to structure it as shown in Figure 2:

2

The red arrows point to the folder for the Event Store and the Working Folders for Usage Analytics, Search Analytics, and Search Reports. (For more details and hands-on system examples, watch the video from the first chapter starting at time index 4m59s -> direct LINK.)

The big picture about collecting the data from the WFE servers to the Office Server folder is shown in Figure 3:

3

To complete the picture, you need one more part, which is, however, not relevant to the current discussion. Just be aware that a lot of data collected by the usage data collection mechanism is logged to the LoggingDB database.

To get a complete overview of what is collected by the usage data collection mechanism, you can go to Central Admin -> Configure usage and health data collection. Or you can use the PowerShell call: Get-SPUsageDefinition.

The output of that call shows all usage definitions. The interesting parts for analytics are Analytics Usage and Page Requests, which you see in Figure 4:

6

Timer Jobs

Timer jobs for analytics can be separated into two different groups. One group is for collecting data, and the other group is for cleanup data. The groups are:

Usage Analytics:

  • Usage Analytics Timer Job for Search Application Search
    • Periodically schedules processing of the Usage Analytics analysis
  • SharePoint Foundation Usage Data Processing
    • Checks for expired usage data at the farm level and deletes the data
  • SharePoint Foundation Usage Data Import
    • Imports usage log files into the event store

Search Analytics:

  • Analytics Event Store Retention
    • Periodically cleans up the Event Store and the Reporting Database
  • Analytics Timer Job for Search Service Application
    • Periodically schedules analytics for Search Service Application

PowerShell for Analytics

First , note that the PowerShell snapin for Analytics is not part of SharePoint common PowerShell snapin. To work with the Analytics engine, you have to register the needed snapins as Figure 5 shows:

#Set up the Anaytics powershell environment
Add-PSSnapin Microsoft.SharePoint.PowerShell
Add-PSSnapin hostcontrollerpssnapin
Add-pssnapin junopssnapin
Add-pssnapin searchcorepssnapin
Add-pssnapin enginepssnapin
Add-pssnapin analysisenginepssnapin
$env:CERES_REGISTRY_PRODUCT_NAME = "Office Server\15.0\Search\Ceres"
Connect-System -Uri (Get-SPEnterpriseSearchServiceApplication).SystemManagerLocations[0] -ServiceIdentity (Get-SPEnterpriseSearchService).ProcessIdentity
Connect-AnalysisEngine -NodeName AdminComponent1

 Now you can interact with the Analytics engine. The cmdlet Get-Command -Module analysisenginepssnapin gives you an overview of what can be done. Indeed, it’s a good idea to understand what is going on during an Analytics run by looking at the definitions of the different Analytic types, which Figure 6 shows:

#Get settings for UsageAnalytics
Get-AnalysisConfiguration usageanalytics

#Get settings for SearchAnalytics
Get-AnalysisConfiguration searchanalytics

#Get settings for SearchReports
Get-AnalysisConfiguration searchreports

Using Set- AnalysisConfiguration you can manipulate the setting. An example that can be very useful during debugging or developing against the Analytics component is to disable the binary and the compress format for the files in the Working Folder (for hands-on system examples, watch the video from the first chapter, starting at time index 6m26s -> direct LINK):

7

Another useful command is starting an Analytics run manually (for hands-on system examples, watch the video from the first chapter starting at time index 7m49s -> direct LINK):

Start-Analysis UsageAnalytics 

To get the actual status of all Analytics components, you can use:

Get-Analysis

Conclusion

This article has given you the basics of data collecting, files and folder locations, timer jobs, and PowerShell scripting for analytics. Be sure to watch the video links I’ve provided throughout the article to give you hands-on examples of working with these analytics features and functionality.

I’ll end this article with a WARNING: Changing settings can cause the engine to fail. Do not change settings you don't understand.

The next chapter will be about Search Analytics feature. In that article, I will talk about analyses in search analytics, FLOWS, ranking models.








Topic: SharePoint Search 2013

Sign in with

Or register

  • Hi there,


    Have you considered a third party analytics solution for your SharePoint portal? A solution like CardioLog Analytics drills down deep to provide much deeper insights than those available in SharePoint or Google analytics.Its available as an On-Prem or SaaS deployment. The solution is specifically built for SharePoint, so its tailored to provide analytics specifically on SharePoints architecture. If youre looking for more information regarding your portal visitors, navigation paths, search behaviors, and content performance check out http://www.intlock.com


    Hope this helps!
  • There are 15 timer job definitions in the SharePoint Server Search OOB. One of my Environments only have 4 of them.
    Is there any way to get Sharepoint to re-create missing default timer jobs?

    I found this problem as I tryed to get some Analytics data, but all data was just blank. 2 of the missing timer jobs are Analytics Timer Job for Search Service Application and Usage Analytics Timer Job for Search Application, both needed to get the Analytics data into the Analytics database.
    The folders you talk about is there, and there is data in them.
    I would love too if I did not need to deletee and recreate my Search Service Application. And doing a backup/restore of it would result in getting the same problem back is my guess.

    Sharepoint Server Search as it should look like:
    Analytics Event Store Retention
    Analytics Timer Job for Search Service Application
    Crawl Log Cleanup for Search Application SP_Test Search Service Application.
    Health Statistics Updating
    Indexing Schedule Manager on
    Prepare query suggestions
    Query Classification Dictionary Update for Search Application.
    Query Logging
    Rebalance crawl store partitions for
    Search Custom Dictionaries Update
    Search Health Monitoring - Trace Events
    Software Quality Metrics reporting for Search
    Spelling Customizations Upgrade
    Spelling Dictionary Update
    Usage Analytics Timer Job for Search Application

    SharePoint Server Search in my faulty Farm

    Indexing Schedule Manager on
    Prepare query suggestions
    Query Logging
    Software Quality Metrics reporting for Search