One of the most common requirements in any SharePoint environment is the aggregation and summarization of content across lists, libraries, sites and site collections. Administrators could not easily achieve these requirements without custom code until the introduction of the Content Query Web Part (CQWP) in SharePoint 2007/2010. The CQWP gave administrators a useful out-of-the-box method for collecting data from various lists into a consolidated view. Even better, it could aggregate data from different sites – something the built-in List View Web Parts could never do. But, the CQWP certainly has its limitations – the number of sites it can query is limited, modifying the display requires expert XSL transformation skills, and the behind-the-scenes heavy lifting required to execute its queries can place a significant load on the infrastructure, especially when used on heavily trafficked pages.
Recognizing these limitations, Microsoft introduced the Content Search Web Part (CSWP, sometimes referred to as “Content by Search”) as part of the on-premise release of SharePoint 2013 and eventually in Office 365 tenants. By this point, most customers should at least be familiar with the CSWP and how it functions. The real question now is which one should you use – Content Query or Content Search? The answer, of course, is it depends. The following is a collection of basic guidelines to help you make an informed decision. For more information on how to use and extend each one, refer to the Additional Information section at the end of this post for related articles, presentations and blog posts.
The fundamental difference between the two Web Parts is that the CQWP issues CAML queries to pre-defined content targets whereas the CSWP retrieves data by issuing queries against the Search index. These are two completely different approaches to content aggregation and consequently the capabilities and functionality of each is unique. Which one you use depends upon your particular requirements and configuration so it's worth a brief review of how each Web Part works.
The CQWP obtains data by interrogating a set of lists and libraries using Collaborative Application Markup Language (CAML), which is the declarative glue that pretty much holds SharePoint together. CAML is an extension of XML and it looks much the same – to get all the items in a list with a value of “Texas” in the “State” column requires a string of text like:
<Query><Where><FieldRef Name=”State”><Value Type=”Text”>Texas</Value></FieldRef></Where></Query>
Although it may not immediately be apparent when configuring the Web Part via the visual designer, the options selected ultimately boil down to one or more CAML queries being executed when the page is loaded. This has the distinct advantage of providing fine-grained control over the result set but it also limits the query to metadata properties alone.
The CSWP, on the other hand, uses the SharePoint Search service as its content source. Rather than querying lists or content types individually, a search query is constructed then passed on to the query components and search index for processing, resulting in a set of content that satisfies the search criteria (such as “?k=State:’Texas’”). This makes it easy to configure and expands the data source to include items that may have the specified keywords or phrases in a block of text or document contents. While the CQWP is very targeted and specific (structured, if you will) the CSWP is more free-form or unstructured, permitting the possible inclusion of a wider range of results.
Perhaps the biggest differentiating factor between the two that heavily influences the decision process is current data relevance. The CQWP executes queries against the target data set(s) in real-time whenever the page is rendered (aided by some background caching processes). This means that as soon as a user adds data to a list it is available for inclusion in the result set when the cache is refreshed. The same applies to changes in metadata – add a new column, update some list items, include the new column in the query definition, and the user will see the results in near real-time. Furthermore, CAML queries can return results for items in an unpublished state, which can be helpful when attempting to aggregate in-progress or pending content.
The CSWP does not have the same capabilities. Because it relies on the Search infrastructure, the data it returns is only as valid as the latest index copy. Depending upon your configuration, this could be as little as fifteen minutes or as long as twenty-four hours (or just about any other time span defined in the crawl configuration). Additionally, if new columns are added, they first need to be indexed, then promoted as managed properties, then the content crawled again for inclusion in the index before updates to the query logic will product additional results. This requires a fair bit of configuration and, of course, administrative access to the Search configuration for the farm or site collection.
Based on this alone it would appear that the CQWP is the most logical choice for data aggregation. Not so fast. Although it is more “real time,” due to the fact that the CQWP is reliant upon CAML queries, the execution of which can place a potentially significant load on the system, and bound to the site collection in which it resides, the amount of data it can gather is limited. The CQWP is also restricted to the metadata fields themselves – content in rich text fields or the body of documents itself is not included. That may severely impact the quality and quantity of information displayed to the user, resulting in a sub-optimal experience.
The CSWP doesn’t have any such limitations because its data source is the much broader set of results included in the search index. This means more data can be gathered from more locations and even include text in Word, PDF, or other document types. That’s a big advantage, especially when aggregating across a deep or wide taxonomy; however, this is offset by the fact that the data may not be as fresh and configuring it takes additional effort.
So the first decision points revolve around the amount and type of data, current relevance of results, configuration overhead and permissions (users without administrative access cannot make the necessary changes to the search properties and crawl schedules). When deciding which to use, it is necessary to have an in-depth conversation with the content owners as to what type of information they need, when they need it, and how they are going to facilitate future changes. There is no “correct” choice – it all comes down to requirements and use cases.
The second primary area of comparison is performance. While it may be desirable to include as much data as possible, what it takes in terms of system resources to retrieve that data can be a major factor. A large data set that, although relevant, adds ten or fifteen seconds to page load times is unlikely to be acceptable to users. In on-premise scenarios, this may be resolvable by adding or enhancing infrastructure components (servers, CPU, RAM, disk, etc.) but even then, there are core platform limits that at some point introduce diminishing returns.
This can be a significant decision point and there is only one clear-cut answer: search. The CSWP vastly outperforms the CQWP by leveraging the SharePoint search infrastructure, which stores all the crawled data in a highly optimized set of isolated databases. Search queries place marginal load on the system and scale well beyond the limits of CAML. It is highly unlikely that a search query will result in a throttling scenario, and the more focused the query, the better and faster the result set (see http://office.microsoft.com/en-us/office365-sharepoint-online-enterprise-help/manage-query-client-types-HA103639818.aspx for throttling types). In addition, Office 365 tenants benefit from continual performance improvements in the CSWP such as the Group Cache feature, which allows query results to be assigned to a group of users and stored in memory for all members of the group (for more information on Group Cache refer to http://blogs.technet.com/b/helgesolheim/archive/2014/03/19/new-group-cache-for-the-content-search-web-part.aspx).
Consider the following scenario: ten subsites each have a custom list with around 1,000 items in each list and the top three most recent items from each list must be aggregated into a master list at the root of the site collection. Retrieving this data using the CQWP would require CAML queries to be executed against each list. Each query would need to do some fairly intensive processing to retrieve the collection of list items, sort by modified date, and select the most recent items. This process would then be repeated ten times before the results could be returned to the user. This process places significant load on the system, compounded by other processes currently running, number of concurrent requests, other page components, and so on.
The CSWP handles this scenario in an entirely different manner. Assuming that the list items are already in the search index, a single query returns the relevant results without needing to access multiple sites, open and dispose of memory-resident objects, or iterate item collections. The overhead and impact on system resources is minimal. More importantly, the result is extremely fast compared to interfacing with the object model APIs. The tradeoffs for these gains, as mentioned previously, are result relevancy and administrative configuration; however, when performance is paramount, there is no question that content aggregation using search is the best answer.
It is also worth mentioning that the excessive use of Content Query Web Parts can lead to severely reduced performance in Office 365. Microsoft is understandably concerned about the impact each customer is having on the overall system; place too great a load on system resources in a multi-tenant environment and other tenants can be negatively impacted. To mitigate such scenarios, Microsoft may impose throttling limits on specific tenants. If your tenancy is making extensive use of Content Query Web Parts, you should give serious consideration to converting these to content search wherever possible in order to avoid a potential throttling scenario.
Another important consideration is the ability of site administrators to modify the query and display options of each Web Part to meet the business requirements. The ability to query data stored in multiple lists across several sites is a fine thing but if the input and output options are inadequate then users are not likely to be pleased with the results.
Both Web Parts provide a wide variety of customization options. For the construction of queries, the CQWP provides options for the selection of a number of inputs, including sites, lists, content types, navigation terms, and field-based filtering; however, changes to these settings must be applied and saved before the results can be viewed. The CSWP, on the other hand, has a live results pane that displays a sample set of results in real-time. This is a handy time-saver and helps with the construction of complex queries, especially in advanced mode where the query can be manually edited.
One significant drawback to CSWP customization is the need to understand, at least at a basic functional level, how the search query language works. Attempting to retrieve column values that have not been included in the managed properties configuration won’t return the expected results and the use of refiners is a complex topic in and of itself. The CQWP obfuscates the actual query in favor of a more user-friendly, point-and-click approach. Which is preferable is largely determined by the sophistication of the individual and their permissions.
Neither Web Part is clearly superior when it comes to customization. Both have a wide array of options and demand a certain level of knowledge and skill. For overall ease of use, the CQWP likely has the upper hand due to the relative ease of selecting various filters and inputs; however, the power of the search query language and the refiner capabilities give the CSWP and edge in fine-grained control. Which to use in a given scenario will vary based on the relative sophistication of the user and access to developer resources for output modification.
In terms of overall capabilities and customization options, both the Content Query and Content Search Web Parts give users a powerful set of tools for the aggregation and display of data. The major area of differentiation is performance, especially the ability to manipulate large amounts of data with minimal impact on system resources. On this point, the CSWP is far and away the best choice, especially for Office 365 customers. This makes content search the most applicable solution across a wide range of workloads. Although capabilities, resources and requirements will vary from customer to customer, the use of the Content Search Web Part is most likely to result in the most flexible and performant option in most scenarios.