When teaching about data warehousing (DW) and business intelligence (BI) fundamentals to IT groups, business people or students at a university I always mention the failure rates of these projects that are often quoted. Those number are always over fifty percent and often much higher. If the failure rate was really that high then I doubt so many people at so many companies would be using DW/BI to operate and manage their businesses. The question arises as to what people mean by failure.

Most DW/BI projects are completed and business people use the resulting information. Clearly failure in this context does not mean that the technology used and the applications build are not working. We are not talking technical failure such as your dishwasher or cell phone not working. But many BI/DW projects have two traits that would lend credibility to the term failure.

First, many BI/DW projects are late and over budget. They get completed but the time, resources and costs are greater than anticipated. People often underestimate the level of effort to get requirements; have the business people agree on definitions, transformation and metrics (the more groups involved the longer it takes; develop data integration that includes cleansing and making data consistent; and, managing scope creep during BI (dashboards, reports and analytics.)  Depending on the amount of the miss in schedule, resources and costs, one would not five the BI/DW team an F grade but maybe a C- or even a D.

Second, and of more concern, is that BI/DW projects often fail to meet expectations. Everything might have gotten done as the IT group, systems integrator or software vendor said but the business just does not feel it was as promised. Maybe the expectations were that many more business people would have found the resulting BI reports/dashboard/analytics easier to use or of more value. So the shortfall is that only a small portion of the expected business users are customers of the BI solution. Maybe the business people expected a lot more analytics built for them rather than reports or “roll your own” analysis (self-service had been the industry buzzword for this.) Or maybe some of the critical information that the business was expecting has not become available or is not clean enough to use with the BI solution. I might give them a D+ on this but the business groups who paid for the BI project would porbably be inclined to give grade of F.

The usual suspects for this expectation shortfall is often shared by many people: the software vendors or systems integrator (SI) who were a little over zealous with their commitments; the IT group who were a little inexperienced or did not do enough due diligence with the vendor or SI promises; and, the business people who were naïve enough to believe the claims proclaimed in PowerPoint slideshows from sales people. If it looks too good to be true, it probably is.

It would be great if all it took was buying the right tool and your business would get comprehensive, cleansed, consistent and current business information from data created from across an enterprise and from interactions with customers, suppliers, partners and stakeholders.  You do need to buy tools but it does not happen in minutes, hours or days. It takes time and hard work.

For your next DW/BI effort do your due diligence, understand what it will realistically take to deploy what the business wants and, above all, set the proper expectations. I have heard many people tell me that is not what the business wants to hear but let me tell you the pushback in the beginning is nothing to what will happen if you fail.  And it does not matter how much you feel has been delivered, it only matters what the business, your customer, feels happened.

Fyi: The picture is of Tom Brady walked off the field at the end of this year’s Super Bowl where the New England Patriots failed to meet expectations. A perfect t regular season and many football records broken but the Patriots in the eyes of many fans and players failed because they lost one game.