Gartner Research published its "Magic Quadrant for Data Integration Tools, 2006" on November 22, 2006. It has been eighteen months since they published the “Magic Quadrant for Extraction, Transformation and Loading, 1H05”. Gartner has decided to supplant its ETL (Extract, Transform and Loading) Tool Magic Quadrant with the Data Integration Tool Magic Quadrant because “Discrete markets for single-purpose data integration tools, such as extraction, transformation and loading (ETL), have given way to a broader market for data integration tools that address a range of data delivery styles.”
Historically, the product categories of ETL, data federation or enterprise information integration (EII), enterprise application integration (EAI) and data replication were considered discrete markets. As these products matured and their use expanded, particularly with the widespread use of ETL for data warehousing, people started looking at data integration holistically and realized that ETL, EAI and EII could really be viewed within a single architectural framework. In addition, a data-integration framework includes data profiling, data modeling, data profiling and metadata management. These product categories have also been developed and sold as separate entities.
Even though Gartner classifies data-integration tools as being in an “early stage” market they felt they should shift their analysis from ETL only to the more expansive data-integration market. I applaud their decision to provide an analysis of the data-integration market and feel it provides more value to enterprises who share the data integration vision. However, in a subsequent post I will discuss why ETL market analysis still offers significant business value.
The data integration market leaders profiled in the report are not a surprise to anyone following this industry. Informatica and Ascential (now part of IBM) have been two enduring leaders for the ETL Magic Quadrants since May 2002 and now are the only occupants of the data integration leader quadrant. These two companies have certainly led the market in mindshare and execution for years.
But more importantly both companies have pushed the data-integration envelope by laying out an architectural vision and developing product roadmaps to achieve that vision. Their roadmaps have involved acquisitions as well as organic development of their own products. Ascential’s history has involved not just acquiring companies, but they themselves were acquired. And they were acquired twice! (They were first acquired by Informix, which then sold off the database portion of that company and they were subsequently acquired by IBM.)
With the switch from an ETL to a data-integration tool analysis, many of the software vendors that were examined eighteen months ago moved out of or further away from the leaders’ quadrant. Several software vendors have very worthy stand-alone ETL tools but they are farther from providing the more extensive data-integration capabilities than the leaders.
After the leaders, the rest are split into the following quadrants:
It will be difficult for any companies to break into the leaders’ quadrant unless the overall market for data integration expands significantly (below.) Products from the market leaders, Informatica and IBM, are being used successfully by many companies for their data warehousing, business intelligence and performance management efforts, often on a daily basis. In many cases, these solutions have been used for years and an extensive code base and associated expertise has been built up in those companies.
It would not seem to be a persuasive business case to supplant those products with any others. In fact, both of the leaders have evolved their ETL products to expansive data-integration platforms supporting further expanded use of their products in their existing clients’ environments, as well as future prospects. Often times, software companies can get their existing clients to purchase new product categories from them, however, in the case of data integration the BI and database vendors find that their largest clients are already customers of Informatica or IBM data-integration products and generally have standardized on them. Unless these vendors can make the data-integration pie bigger then they may not be able to break into the leaders’ quadrant.
It is interesting that the two niche vendors “on the cusp” of the challengers’ quadrant are the software giants Oracle and Microsoft. Both offer databases, middleware, applications and other products. Both have also significantly upgraded their data-integration capabilities. Microsoft’s release of SQL Server 2005 with Integration Services has gone a long way to enhance its previously very rudimentary capability (and that’s being very generous with regards to DTS.) Microsoft Office PerformancePoint Server 2007 will further extend their data integration framework. Oracle smartly enhanced its ETL functionality in Oracle Warehouse Builder (OWB) 10g Release 2 but its acquisition of Sunopsis should greatly enhance its data-integration capabilities.
These are the two vendors most likely to be able to expand the data-integration market, especially because of their presence in the SMB market along with leveraging their enterprise application business.
I enjoyed your analysis.
I graphed the movements in vendor rankings from the ETL quadrant to the data integration quadrant in my blog entry Oracle Plunges and no one soars in the 2006 Gartner Data Integration Tools Magic Quadrant.
Oracle and Ab Initio were strong in previous quadrants but suffered big drops in this one due to lack of a solid metadata strategy. The Sunopsis purchase still doesn’t give Oracle much in the way of profiling, metadata or data quality so they are still a long way off making the leader quadrant.
SAS and Business Objects both have a good shot at the leader space as long as they can keep pumping out new functionality. They need better product integration and wider scope. Microsoft is hampered by the very long gaps between major releases. They could easily make the leader quadrant if they put enough energy into it. They seem to be quite serious about ETL and data quality but not so strong on metadata and profiling.
I enjoyed your analysis too. I especially liked how you graphed the movement of the vendors between the different versions of the Gartner Magic Quadrants.
I agree with your assessment of vendors in your comment. Let me add:
Ab Initio also suffers from their “close to the vest” strategy.
Oracle buying Sunopsis hopefully will turn out well IF they “merge” the products. WB 10g R2 did have the start of data profiling and data quality functionality. With Sunopsis you can get to the metadata but you should not have to “roll your own”. Since it owns many enterprise apps now, you would think Oracle would be in a great place to develop a solid metadata mgt offering but…
Business Objects and SAS do have a shot at the leaders’ quadrant from a technology perspective but maybe not gain significant market share along the way. I enjoy using and think highly of Business Objects Data Integrator and like their EIM positioning but will that result in significant sales? Their sales growth may become large but those comparisons are from a small installed base (as compared to the leaders.)
You would think that Oracle have enough data management, integration and middleware technology for a good metadata product, but that volume of applications works against you as you end up creating and maintaining huge numbers of metadata brokers. Fusion is the place for a good metadata server and services layer if they haven’t already missed the boat.
IBM-Ascential learnt the hard way that you are better off creating your metadata server and putting your products on top of it instead of the other way around.