The good news is that enterprises have recognized the significant business value of analyzing the surging amount of information and then
acting upon that analysis. The bad news is that many enterprises have become overwhelmed by the information deluge and either cannot effectively analyze it or cannot get current enough information to act upon.

Enterprises are in this position because many are still using the standard techniques and technologies that became mainstream in business intelligence (BI) and data warehousing (DW) before the current information deluge. Enterprises have been encountering significant increases in volume, variety and velocity over the last several years as they have expanded from integrating data from internal operational systems to data interchange with customers, prospects, partners, suppliers and other stakeholders.

Although businesspeople do have considerably more data available, there is often an information gap with data locked in data silos and spreadsheets being used as a sort of data superglue. Businesspeople are frustrated with lost opportunities to better manage their business or grow their enterprise’s revenue and profits.

What is Holding Enterprises Back?

BI and DW has matured over the last two decades, enabling enterprises to gather and integrate data from their internal operational systems and external data sources. The typical technical architecture used includes servers and storage coupled with relational database, data integration and business intelligence software. The capacity and throughput of these systems have expanded over the years but these systems have been general-purpose systems built to handle transactional, operational and BI applications. The current information deluge is stress testing the underlying assumption that general-purpose hardware and software can meet the demands of business analytics that enterprises need today.

The typical data architecture for BI involves creating a DW where all data needed for BI is integrated and then building special purpose data marts or OLAP (online analytical processing) cubes designed for and tuned for specific business analysis. This architecture has been effective, but many enterprises are unsuccessful in keeping up with the information deluge, because the process is time-consuming, labor-intensive and often requires skills that are stretched or in short supply.

The technologies supporting business analytics, DW and BI have been evolving over the years. Technical innovations have greatly expanded server speed, memory capacity, network bandwidth and storage access speeds. In addition, relational databases leveraging these innovations have increased their capabilities. But will all these advancements, enterprises are still falling behind because much time and many skills are necessary to design, build and tune the many BI specific databases, cubes and data structures necessary to enable the analytics needed by businesspeople today. And while these are being built, many new applications and data shadow systems are being created because the information demand is not being met, further exacerbating the data silos problem in enterprises.

What Should We Do Differently?

The current wave of emerging technologies for analytics and data integration are addressing the current shortfall by concentrating on reducing the time to build analytical applications, supporting greater data variety and analytical complexity, reducing many of the tool-specific skills that are in short supply and lowering the total cost of ownership (TCO.)

Business analytics has specific data usage and workflow patterns that are quite different than transaction processing systems. Much time and skill is needed by enterprises in implementing BI using the typical technical and data architectures. BI appliances, columnar databases, massively parallel processing (MPP) and in-memory analytics have incorporated the analytical data and workflow patterns into their design – often significantly reducing the need by enterprises to manually build these capabilities into typical system implementations.

BI appliances appear most likely to be the disruptive technology innovation for business analytics that enterprises are looking for. BI appliances may include hardware redesign and incorporate a combination of the emerging software innovations such as columnar databases, MPP and in-memory analytics.

Some BI appliances have redesigned hardware such as specially designed chips that handle analytical I/O and compression techniques to increase capacity and throughput. By incorporating this capability into hardware speeds up business analytical processing even before it hits a database or disk. This means that a business person’s dashboard or other analytical application is quicker “out of the box” with appliances.

The most effective BI appliances have gone further than these hardware advances and have reduced the reliance on relational databases to enable business analytics. Columnar databases, MPP and in-memory analytics are all designed to speed up processing and reduce (or eliminate) the time necessary to build specially designed and tuned relational databases or OLAP cubes to support BI behind the scenes. These emerging technologies enable business analytics by supporting the onslaught of variety, volume and velocity of data. Businesspeople always wish to examine data in new ways to meet changing market conditions and customer needs. These technologies enable self-service business analytics by eliminating reliance on newly designed and tuned BI data structures every time a businessperson wants to examine data in a different way.

Although there are many highly skilled data architects and database administrators in enterprises today, there are still not enough to keep up with business analytics demand. BI appliances promise to lessen the needs for the skills required to create and tune the BI databases and OLAP cubes necessary today. As an aside, this does not mean enterprises do not need these architects but that they can shift their skills to incorporating the data volume, velocity and variety that enterprises need to include in their business analytical platforms via BI appliances.

A word of caution when examining and evaluating the use of BI appliances to implement an enterprise’s business analytical needs. Look under the covers to determine what the true innovation is, both in terms of hardware and software, in the BI appliance’s architecture. There are a few BI appliance pretenders whose vendors have merely bundled existing hardware and software into a product offering using the BI appliance label. Make sure you understand what is being offered in the BI appliance architecture, how it addresses your business analytical needs, the impact on speed to implementation and how it lowers TCO.

BI appliances offer to be a disruptive force that may enable many enterprises to catch up and then stay ahead of their business analytical needs.