What is Data Integration?

The cloud as we know it is here to stay. Given how quickly it has been developed, deployed, and adopted, coupled with its pay-as-you-go pricing model, it’s no wonder why so many of today’s top organizations and new entrants are continuing to embrace all the cloud has to offer. However, as a data management professional, there has always been one thing I felt was missing from the cloud: Data Integration.

Simply put, data integration encompasses how data moves between systems. For example, suppose you have a Instagram account that is connected to your Facebook and Twitter accounts. When you post a picture to your Instagram account, you have the option of also posting that same picture to your Facebook and/or Twitter accounts as well without separately logging into those accounts and reposting the same picture. This, my friends, is basic data integration in a nutshell. However, let’s think about if from the cloud standpoint.

In the cloud, every service offering breaks down into 2 essential pieces: storage and compute. That’s it! And if you think about it, all of computing as we know it, cloud or not, revolves around these 2 key ingredients.

Now, knowing that the cloud is rooted in compute and storage, you could imagine that developers simply create programs or API’s that connect systems and move data between them. You’d be right in thinking this; it happens on a majority of data integration use cases. So, if data integration is happening today, where is the gap? In a word: Transformations.

Transformations, as it relates to data integration, deals with how data should be “massaged” (prepared and validated, for example) while it is being transmitted between systems. Most of today’s modern programming interfaces allow for easy connections to be made between systems, however there is a clear gap between connecting to a system and preparing the data so that the systems can interface with the data. This fundamental gap lies in skillsets (between app and data developers) as well as technologies (API’s versus data integration tools). In my opinion, this represents the data integration tools gap in the cloud today, and is something that I have often felt has not been closed in a long time…until now.

The integration Platform as a Service (iPaas) represents a niche cloud architecture that solves for a specific demand in the marketplace that is too broad for the traditional cloud Platform as a Service (PaaS) architecture to handle.

Specifically, iPaaS platforms allow for systems to not only connect with each other but also add a layer of visibility into sophisticated transformations, validations, and any other manner of massaging data during transmission. Even better, these platforms are built for both app and data developers, thus closing the skills gap and allowing both parties to develop as they were trained.

Working with Boomi in recent months has lead me to realizing that many of the gaps I noted earlier are being closed and that those of us in the data management space are finding that our skills around data integration and transformations are being used in new ways in the cloud. Where this platform really excels is in a couple of areas:

There are many other areas where this platform excels, but at the end of the day, the platform really works to close significant people, process, and technology gaps that have been present in the cloud data space for many years.

No way! In this great, big arena of cloud and its ever increasing velocity of services, the chances of finding one solution that does everything for your organization is very, very slim. While iPaaS platforms like Boomi offer a complete data integration solution in the cloud, there are other aspects of the entire data lifecycle we have to take into consideration — think data governance as it relates to data quality.

Nevertheless, iPaaS does represent a huge leap forward in creating a more holistic method of next generation data integration in the cloud.

Related posts:

Given the recent demise of Level Money, I was forced to search for a new personal spending tracking app. After some very brief Googling, I discovered Penny. Penny brands itself as not just a…

Cloud is one of the most talked about thing in the world right now. From our memories to our work files, most of our items are being stored in the cloud each day. Already, services such as Netflix…

Taukah apa itu WordPress? Wordpress adalah sebuah aplikasi open source yang sangat populer digunakan sebagai blog atau bisa disebut juga CMS ( Content Management System ) atau pengolah konten blog…