Business Intelligence & Data Warehousing Trends for 2008 - Data Shadows Systems Continue to be Pervasive for Reporting and Analytics

I co-authored the TDWI report Strategies for Managing Spreadmarts: Migrating to a Managed BI Environment. In a survey we conducted in the fall of 2007 we found that the medium number of data shadow systems at an enterprise was 30. According to the survey, data shadow systems are prevalent in all industries, companies of all sizes and throughout various business functions in the enterprise.

In short, they are everywhere!

BI may not be pervasive, but spreadsheets are. And reporting and analysis are prevalent in every business – that’s how the business operates and grows – so that means that people are using spreadsheets to do that reporting and analysis.

Just ask any analyst or manger in finance, marketing or sales how they do their forecast, compare actuals to budget and, in fact, prepare the budget to begin with. Chances are they are using a spreadsheet. And even if they are using a BI tool or application, probe a little more and you’ll see spreadsheets augmenting the tools they are using.

Several trends will continue to foster spreadsheets and the creation of data shadow systems:

• Business people know how to use Microsoft Excel and have it on their desktops
• BI tool vendors have improved Excel integration, ironically resulting in an increase of data shadow systems
• Now that the “major” BI players (Business Objects, Cognos and Hyperion) have been acquired by the software titans IBM (IBM), SAP (SAP) and Oracle (ORCL), they will be in product transition, i.e. product roadmap phase, delaying large-scale rollouts in existing BI customers
• An economic recession, or concerns about an impending recession, will most likely delay or cancel large-scale rollouts
• The Microsoft (MSFT) juggernaut continues, particularly in the small to medium businesses (SMB), to establish a compelling TCO (total costs of ownership) proposition to customers. Microsoft is slowly but surely expanding and converging the capabilities of SQL Server (database, Integration Services, Analysis Services and Reporting Services), Performance Point, SharePoint, Microsoft Office (especially Excel and Pivot Services.) This product set may be looked down upon by the industry “elite” (most likely with ties to the software titans) but it’s the people, i.e. customers, that decide what is best for them (rather than what is the best tool in an esoteric product evaluation.) It’s the cost, ease of use and familiarity that tips the scale in Microsoft’s favor.
• Other lower cost options compared to the software titans, such as on-demand software, open source and some innovative BI software firms will continue to grow, but spreadsheets will be part of the deployment strategy with these tools.
• The growth of performance management applications will also increase the use of data shadow systems because, despite PM vendors’ claims, their solutions in general will spur business people to use data shadow systems to either customize their solutions or be the duct tape that gets the PM solutions to work with their business environment.

In short, “The song remains the same.” Spreadsheets and data shadow systems keep providing the reporting and analysis needs of business people. There may be better ways to deliver those needs, but business people have to operate the business day to day regardless of how elegant the BI solution is around the corner.

(Download the free TDWI report: Strategies for Managing Spreadmarts: Migrating to a Managed BI Environment and view the on-demand webinar.)

Rick Sherman, Athena IT Solutions