Gartner Research published its "Magic Quadrant for Data Integration Tools, 2006" on November 22, 2006. It has been eighteen months since they published the “Magic Quadrant for Extraction, Transformation and Loading, 1H05”. Gartner has decided to supplant its ETL (Extract, Transform and Loading) Tool Magic Quadrant with the Data Integration Tool Magic Quadrant because “Discrete markets for single-purpose data integration tools, such as extraction, transformation and loading (ETL), have given way to a broader market for data integration tools that address a range of data delivery styles.”

Historically, the product categories of ETL, data federation or enterprise information integration (EII), enterprise application integration (EAI) and data replication were considered discrete markets. As these products matured and their use expanded, particularly with the widespread use of ETL for data warehousing, people started looking at data integration holistically and realized that ETL, EAI and EII could really be viewed within a single architectural framework. In addition, a data-integration framework includes data profiling, data modeling, data profiling and metadata management. These product categories have also been developed and sold as separate entities.

Even though Gartner classifies data-integration tools as being in an “early stage” market they felt they should shift their analysis from ETL only to the more expansive data-integration market. I applaud their decision to provide an analysis of the data-integration market and feel it provides more value to enterprises who share the data integration vision.  However, in a subsequent post I will discuss why ETL market analysis still offers significant business value.

The data integration market leaders profiled in the report are not a surprise to anyone following this industry. Informatica and Ascential (now part of IBM) have been two enduring leaders for the ETL Magic Quadrants since May 2002 and now are the only occupants of the data integration leader quadrant. These two companies have certainly led the market in mindshare and execution for years.

But more importantly both companies have pushed the data-integration envelope by laying out an architectural vision and developing product roadmaps to achieve that vision. Their roadmaps have involved acquisitions as well as organic development of their own products. Ascential’s history has involved not just acquiring companies, but they themselves were acquired. And they were acquired twice! (They were first acquired by Informix, which then sold off the database portion of that company and they were subsequently acquired by IBM.)

With the switch from an ETL to a data-integration tool analysis, many of the software vendors that were examined eighteen months ago moved out of or further away from the leaders’ quadrant. Several software vendors have very worthy stand-alone ETL tools but they are farther from providing the more extensive data-integration capabilities than the leaders.

After the leaders, the rest are split into the following quadrants:

• Challengers’ quadrant: Business Objects and Pervasive Software
• Visionaries’ quadrant: SAS Institute
• Niche players’ quadrant: 12 companies including Oracle, Microsoft, and iWay Software

It will be difficult for any companies to break into the leaders’ quadrant unless the overall market for data integration expands significantly (below.) Products from the market leaders, Informatica and IBM, are being used successfully by many companies for their data warehousing, business intelligence and performance management efforts, often on a daily basis. In many cases, these solutions have been used for years and an extensive code base and associated expertise has been built up in those companies.

It would not seem to be a persuasive business case to supplant those products with any others. In fact, both of the leaders have evolved their ETL products to expansive data-integration platforms supporting further expanded use of their products in their existing clients’ environments, as well as future prospects. Often times, software companies can get their existing clients to purchase new product categories from them, however, in the case of data integration the BI and database vendors find that their largest clients are already customers of Informatica or IBM data-integration products and generally have standardized on them. Unless these vendors can make the data-integration pie bigger then they may not be able to break into the leaders’ quadrant.

It is interesting that the two niche vendors “on the cusp” of the challengers’ quadrant are the software giants Oracle and Microsoft. Both offer databases, middleware, applications and other products. Both have also significantly upgraded their data-integration capabilities. Microsoft’s release of SQL Server 2005 with Integration Services has gone a long way to enhance its previously very rudimentary capability (and that’s being very generous with regards to DTS.) Microsoft Office PerformancePoint Server 2007 will further extend their data integration framework. Oracle smartly enhanced its ETL functionality in Oracle Warehouse Builder (OWB) 10g Release 2 but its acquisition of Sunopsis should greatly enhance its data-integration capabilities.

These are the two vendors most likely to be able to expand the data-integration market, especially because of their presence in the SMB market along with leveraging their enterprise application business.