I love a good argument, and I got one from a reader, Michael Gross, when I published Power BI for Office 365: A collaboration tool to drive business insights. As a solution architect for Business Intelligence, Michael has valid perspective and experience to make his cautions worth considering. Here's Michael's article. I welcome your perspectives as well.
Self-service BI is bad!
By Michael Gross
That’s right, self-service BI is bad. First, let’s take a look at a definition of the term. A search on the net gives us this:
“Self-service BI allows business users to work with data analytics and access business critical information without consulting the IT department. This practice requires business users to have knowledge of how to use BI tools to best leverage the corporate information they need in the decision-making processes.”
The rest of the article writes itself doesn’t it?
Now let’s look at the history of self-service BI. It’s not new. Business users have been accessing business data since they have had the tools to do that. Since as early as the 80s, with software like Lotus 1-2-3 and Paradox, users have been compiling and evaluating data on their own. But the above definition states that the business user is required to “have knowledge of how to use BI tools to best leverage the corporate information.” And there lies the problem.
It should also be noted that size of the organization defines to a great extent the value of self-service BI. In the 90s many businesses, particularly enterprise-level businesses, went through a contraction of manpower. Many employees were laid off, and departments were thinned, combined, or otherwise transformed. For the enterprise-level business it no longer made sense to have dozens or more workers spending a majority of their time mining for data.
Centralizing data mining and compilation into teams where the worker was a specialist in the data repositories and the relationships of the data allowed the organization to reduce costs. Their results were then placed in report centers where the business user could consume them. Having had all the hard work done already, all the business user needed to do was evaluate the report and make the appropriate decisions.
Now I recognize that smaller and mid-sized organizations may have fewer options. In many cases, users need to wear many hats in these companies to get the job done. But that doesn’t excuse the problems that can occur from having unskilled data analysts running data models.
What happens when end users do BI
Over the past decade software vendors have been pushing the idea of moving the data mining experience back out to the end user. This reopens all the issues that we had in the past. End users are not the best qualified people for determining the validity of the results of that. End users are not as efficient at mining as are specialists who do it all the time and have a close, clear understanding of the data and its relationships. Here’s an example to explain why:
Company A has 5 financial analysts. They all receive a daily series of reports that contribute to the decision process for their work efforts. 1 of the 5 has independently created a new report and uses this report, along with the others, to build a decision matrix.
Two possibilities exist: First, the report created independently could be valid. In this case, it should be shared with the other 4 analysts for their consumption. Alternatively, the report could be invalid, meaning it is a statistical artifact and should not be used in a decision-making matrix. A centralized data analysis team would be well-suited to know if this is the case.
What is a statistical artifact? Consider this example. The boss asks for a report that shows employees hired over the past 2 years that left within the first year. The report needs to be grouped by ethnicity and the results shown as a percentage of each group. The inexperienced end user does exactly that and provides the following output.
This graph shows that minorities are markedly more likely to leave within the first year of employment. But how valid is this data? Instead of looking at the percentages of individuals, let’s take a look at the raw numbers.
The raw numbers tell a completely different story. This is a case where a thorough understanding of the data would benefit the consumer.
So, self-service BI is bad. Well, usually. It may be valid but it is definitely not efficient for the organization. What is efficient is a dedicated data quality/data analysis team whose sole function is to provide valid, verified business intelligence to the end user for their consumption.
Let the experts do the work
It's not a good idea to have end users spend significant amounts of time data mining and generating reports when they could be using decision-making matrices. When it comes to BI, leverage your strengths wherever possible. Even in the smaller organization it is possible to centralize BI activities for better value.
You could argue that it is a better value to have a lower-cost end user performing the data mining over a more costly skilled resource. It is not. All the value is lost in the amount of time spent on unskilled data mining. You could argue for the use of an accuracy /cost model that says it is acceptable to have reduced accuracy to leverage the lower cost. It is NEVER acceptable to have reduced accuracy when performing business decisions based on BI results.
In fact, it is almost always more cost efficient to have specialists perform job functions that require unique skills at a higher per capita cost than unskilled workers at a lower cost.