Data Integration: Beyond ETL and Data Warehousing

Business Intelligence (BI) Index: Weekly Update 06-26-2009
June 29, 2009
Pervasive Business Intelligence Solutions
July 1, 2009
Show all

Data Integration: Beyond ETL and Data Warehousing

published in Information ManagementData integration suffers from an image problem. It has become synonymous with extract, transform and load (ETL). Likewise, ETL has been regarded as a data warehousing (DW) technology.  Both of these viewpoints do not reflect current capabilities and greatly inhibit enterprises in their attempt to integrate data to provide the information their business needs.

Because of this short-sightedness companies have lost opportunities to harness information as a corporate asset. It increases the cost of integrating data, encourages the creation of data silos and forces business people to spend an inordinate amount of time filling the information gaps themselves through data shadow systems or reconciling data.
>>> Continue to the rest of my article, Data Integration: Beyond ETL and Data Warehousing, on the Information Management website.

1 Comment

  1. Rick,
    A very interesting article that I agree with in large part. Data integration does suffer from a simplistic association with the ETL of yester-year. Yet data integration is so very much more and has, as you mention, moved forward considerably. I believe that data integration has come of age as organizations look for alternatives to the traditional headache of maintaining hand-coded interfaces between applications. It is interesting to consider that all of the main business imperatives facing organizations today rely upon data integration to achieve them – For example, business intelligence requires access to trusted data (that’s data integration); Regulatory compliance requires fast access to trusted data (that’s data integration); Mergers and Acquisitions require combining data together for regulatory purposes or defining a single view of customers for increased cross-selling and up-selling (that’s data integration); acquiring new customers and retaining customers requires a better understanding of customer data through a single view of customer (that’s data integration); outsourcing non-core functions to the cloud requires data to remain synchronized with on-premise data (that’s data integration); movement of data between a client and it’s partners requires interfaces built on emerging XML standards like SEPA, HIPAA or HL7 (that’s data integration).
    Today we are faced by an IT landscape that has focused on building infrastructure and automating business process department by department. Over time we have seen higher percentages of our IT budgets spent on simply keep the lights on. Part of this is due to the inefficient ways in which we have hand-coded application integration to deal with the problem of data inconsistencies. A modern data integration platform allows you to automate the interfaces between applications, but also reduce the KTLO budget by archiving data out of operational applications, subsetting datasets for development processes and quite simply cutting large parts of budget out of the system. The evolution of data integration is analogous to that of databases 20 years ago (when we moved from building to buying) and backup 10 years ago (when we moved from hand-coding scripts for different machines to buying commercial applications). The same is happening around data integration as organizations see the benefit of using commercially available solutions rather than dealing with the hand-coded maintenance nightmare.
    I would go so far as to say that data integration is the single most critical technology platform since it helps reduce IT costs and gives organizations a critical competitive advantage by empowering them to access, integrate and trust their information assets whether they be within traditional enterprise applications, fragmented across analytical systems such as spreadsheets and PDF”s, in the clouds or exchanged between trading partners. Indeed in the modern information society we live in, data is king and data integration is the enabling technology to allow businesses to gain the greatest value from their data assets.
    Wow – how far from ETL have we come!

Leave a Reply

Your email address will not be published. Required fields are marked *