This month, we explore the impact of several business intelligence (BI) trends on data integration. These trends include: business (or corporate) performance management (BPM or CPM), real-time analysis, dashboards and data visualization and, surprisingly, reporting. The common thread through all of these initiatives is that data is the foundation of their success. Dashboards, data visualization and analytics are just tools and, as such, are not very useful without the data. Yes, these tools are often presented as the solution for data silos; however, they are merely the visual representation of the data integration that lies underneath the covers and beyond the proof of concept.

BI Trend: Performance Management

The BI trend with the most press and marketing hype is performance management. BPM, as offered by vendors, is a combination of BI tools, pre-built analytics supporting metrics or key performance indicators (KPIs), pre-built databases containing the data to be analyzed, ETL (extract, transform and load) tools and pre-built data mappings (ETL applications) to source the data for those databases. Vendors’ solutions are pre-built stacks of tools, data, applications and often services to accelerate the time for a company to deploy these solutions. The vendors offer two value propositions in tight economic times: it’s cheaper and faster to buy versus build, and it is one-stop shopping that offers reduced license fees and fewer integration points.

The most obvious impact is that BPM needs data, and the vendor offerings bundle that into their solution. On the positive side, this raises the awareness for data integration and ensures projects are using ETL tools and data integration processes in providing data to the business. On the cautionary side, one must examine how the solution stack – products, technology and data – integrates with your existing data warehouse (DW), BI and ERP efforts.

What vendors are offering BPM solutions? ERP and BI vendors have both entered the market as natural extensions to their existing markets and will probably dominate the market as it grows. The BI vendors, who have gone through a significant market-consolidation phase that enabled them to expand their BI product suites’ capabilities, are now expanding into BPM by acquiring ETL products, developing applications (analytics, pre-built databases and ETL applications) and offering services for implementation.

ERP vendors have gone through a similar consolidation phase to expand their offerings, along with an evolutionary path of offering data warehouses, then BI capabilities and finally BPM to their application footprint. Both BI and ERP vendors offer BPM in a similar manner in terms of technology and data integration. The vendor solutions source data directly from the ERP systems, populate a data warehouse and then build their analytic applications on top of this data. Buying a BPM package appears to be, and is sold as, a better value proposition than a complete custom solution.

However, is that value proposition faster, better and cheaper? You need to examine it with regard to data integration, particularly in an enterprise-wide implementation. You will most likely still need to customize the BPM solution you just bought and possibly build portions on your own. Ask yourself these questions: How closely do the analytics match how our company examines the business? That will depend, but chances are there will be some significant customization. Does the solution provide all the analytics we need? Probably not; you are going to have to build that portion. Most importantly, most corporations are not dealing with “green fields” regarding data warehousing, BI and ERP reporting. Rather, many firms have already built data warehouses, data marts and BI solutions. Under those conditions, the BPM solution you just purchased might become another data silo that the business needs to reconcile with your data warehouse or ERP reporting system.

One approach to avoid yet another data silo and leverage your existing DW/BI work is to switch the source of the BPM solution from the ERP systems directly to your data warehouse. This approach converts the BPM database from another data silo into a dependent data mart with conformed dimensions. This approach uses the data profiling, data cleansing and data integrity that you already built into your data warehouse and ensures consistent data between the BPM solution and your existing work. This involves customizing the BPM solution’s ETL mapping by changing the source from your ERP system to your data warehouse. This is an excellent option for a company that has already invested in data warehousing by delivering a business solution that uses a BPM package as a base of code while still being more cost effective than building entirely from scratch. However, this approach needs to be sold as a necessary step to provide short-term and long-term value from your data integration efforts.

The bottom line is that BPM requires more data and data integration. In order to provide a business solution that produces business value while avoiding the expense of another data silo, you need to plan well, devise an architecture that meets business needs and integrate the data according to that plan and architecture.

BI Trend: Real-Time Analysis

Real-time analysis, driven by operational pressures, is being publicized as a must-have for corporations. These capabilities are being promoted by both BI and ERP vendors, whose tools often directly access the ERP systems’ applications and data. What is the impact on data integration?

First, you are seeing an expansion of the capability historically used in data warehousing solutions into the ERP and operational systems market. BI and ETL products are being used directly on your ERP system or within an ERP BI solution. These tools, with some extensions, can access databases and applications whether they are labeled data warehouses or ERP systems. This extends data integration from its traditional management-reporting audience to operational analysis. It helps break down the technology silos between these worlds and allows people to concentrate on developing business solutions, rather than having to create different architectures for each class of business users.

Second, data integration means getting the data (real-time versus batch, database versus application), transforming it along with other data and finally putting it someplace. The data functions are the same regardless of the technology platform – ETL, enterprise application integration (EAI) or enterprise information integration (EII). Several vendors have recognized this and offer data integration suites with multiple “transportation” layers (ETL, EAI and EII) enabling batch, real-time, database-to-database and application-to-application data movement. This breaks down some of the silos that were created when companies had to choose between overlapping ETL, EAI and EII solutions.

BI Trend: Dashboards and Data Visualization

Dashboards and data visualization are being marketed and deployed as tools to allow people to spend more time analyzing data and less time searching through pages of reports. These tools display summarized and aggregated data, along with highlighting exceptions. The key to the success of these tools is their ability to drill into the details when needed and, hence, increase the need for data integration. These tools are often used in the performance management solutions and display both historical trend data and real-time data when applicable. These tools help create greater demand for data that is timely, consistent and accurate – driving greater need for data integration capabilities.

BI Trend: Reporting is Back

Reporting has made a comeback with BI vendors. They finally realized that people don’t want to start from scratch every time they run a routine analysis. This further expands the suite of BI capabilities accessible to businesspeople, thus decreasing the technical silos that were created when IT had to decide on a single type of BI capability. In the past, you chose a reporting, ad hoc or OLAP product and then built a solution, often a silo, around that product. With current BI suites, the need for different technology stacks disappears, and you can concentrate on the data. This further increases the demands of data integration within an enterprise.

These trends have impacted data integration by expanding data integration capabilities in many ways, reaching into more markets than before and becoming more important with today’s new business initiatives. Vendors sell solutions, but without data integration, you’re only buying more problems.