Summary: Use SharePoint usage reports and user interviews to realize quantitative improvements in search results. The non-technical aspect of search tuning.
Any successful search implementation requires ongoing review and updates throughout the life of the system, commonly called search “tuning.” As users introduce new content, they also remove content and users create new requirements; search requirements are always changing. For that reason, the team managing SharePoint sites should take steps to improve search results on an ongoing basis:
For boxes 1 and 2 above, there are roughly two ways to do this. You should do them both!
First, you should gather search usage statistics from SharePoint to figure out, primarily:
- What are users commonly searching for?
- Which queries are successful and which are not?
If your company uses SharePoint Online, you can find the instructions for finding the reports here. For any other version, a quick Bing search will turn up the instructions for you.
So, without even talking to a user yet, these reports should give you a rough idea of what the targets are, and the quality of the results.
Second, you should add a human element to this process and interact directly with the users to get real requirements and feedback. Report statistics are great, but you also want context. You want to watch users search and see what they’re doing and what they’re thinking!
You can also add a “feedback” button on the search results page. Depending on how fancy you want to get, you can build in a bunch of logic to capture the users’ details, you can determine which browser they were using and more. But, to be honest, though I’ve seen and helped implement several of these, I’ve never seen them get much use. For that reason, I’d call these feedback buttons nice to have but not essential. Remember that if you add such a feature, you need someone available to respond to the feedback.
In the steps below, I describe how I typically help organizations set this program up. You’ll have to figure out exactly what works in your company, but these steps offer a pretty good guideline:
- Identify groups of search users. Find groups of users where all the members will likely have similar requirements and expectations. Perhaps by job function (Engineering, Sales, Legal, etc.), by region, or something else entirely. It depends on your organization. Start by identifying no more than 10 groups and find an enthusiastic representative from each group that is willing to spend a small amount of time providing expectations and feedback around search results for their group.
- For each of these groups, solicit a list of 10-15 representative queries that they might search for regularly. Record each query and the expected outcome. Sometimes the expectations will be unreasonable. Some may reveal a misunderstanding or misuse of the system. You can address these through ad-hoc training sessions. Just be careful not to try to tune the results to meet unreasonable goals.
- Perform each of the queries and determine how well or poorly the current configuration satisfies their requirements (this creates your “before” picture). Do this along with the representative users from the identified groups.
- Utilize the various tools and methods we have available for tuning the search results. Make small, incremental changes to address the requirements. Remember that we’re trying to satisfy 10 sets of users’ requirements. Some adjustments you make to improve one groups’ experience may detract from another’s experience. This is a delicate balance and precisely why changes must be made in small, incremental steps so that the overall impact can be evaluated along the way.
I typically recommend meeting with each group representative to evaluate the results and gather new requirements and feedback every quarter. In the early stages of the processes, it may be valuable to meet with the group representatives more often and then decide on the appropriate cadence.
What I don’t talk about in this article is how to make the adjustments or what tools are available. That’s out of scope for this particular piece, but I talk about some of these methods in my presentation at last year’s SharePoint Conference in Las Vegas. If you want to get a bit deeper into the technical side of search tuning, this article will be a good place to start. I recommend that you do not solicit the personal feedback until you are comfortable working with the set of tools for adjusting search results. Otherwise, you’ll be setting the expectation with users that you may not be able to live up to.
If you want to learn more about the search tuning process, I recommend reading Customizing Search Relevance in SharePoint Server 2007. Though this article was written for SharePoint 2007, it provides a wealth of detailed information about the process of evaluating and tuning search results.
Lastly, the graphic below is one that I’ve shared with customers for years and may prove helpful in visualizing the flow for your own search feedback loop. I’d love to credit the author of this graphic, but I got it years ago and believe it either came from a Microsoft blog or FAST University training document that no longer exists.
I hope you found this article helpful. Happy search tuning!