Some one said that data in a data warehouse is like an old cloth in
your
closet, some time you might not pick it for the next two years. Many of
us have at least 1TB of pocket hard drive that contains lot of data
including musics, photos and videos. How often have you deleted your old
data in your personal hard drive or throw away an old data CD/DVD. I
believe most of us never throw the data considering that the old data
would be useful for us some day. How often has the 'someday' come to
you, may be never till now.
Similar is the case with your business. In many cases, you are forced to keep the historical data due to legal compliance and you never use it. What would you do if you want to use it, how organized is your data for searching, filtering, sorting and presenting it? During the last few months, I was looking into the new demands that were being considered in the Data Integration space and how can it be projected for this decade. The key spaces that needs a good data integration system to take care has been observed as follows:
In short, the data is being captured from every event, action unlike the earlier manual system where most of the data used to be on the books. The volume of data is exploding everywhere and the time to process is reducing (actually the time is constant but the requirement for processing time is low). The data needs needs to be synchronized and made available to all the decision support systems in real time which means real-time data streaming is required to meet the business demand. The data could be residing on the cloud systems such as Salesforce.com or Microsoft Dynamics CRM cloud based systems which are hosted outside of the enterprise. For this to be efficient, this means that the data filtering and interchange should be minimal so that the data interchange time is used most efficiently. The demand for the enterprise IT integration software to select, filter, sort, compress, encrypt and process the data is having new challenges in this era. In simple terms, the integration software is expected to perform at a higher scale that it used to perform earlier.
Similar is the case with your business. In many cases, you are forced to keep the historical data due to legal compliance and you never use it. What would you do if you want to use it, how organized is your data for searching, filtering, sorting and presenting it? During the last few months, I was looking into the new demands that were being considered in the Data Integration space and how can it be projected for this decade. The key spaces that needs a good data integration system to take care has been observed as follows:
- Big Data
- Social Media
- Mobile integration
- Zero downtime
- Demand based real time data synchronization
- Cloud data integration
In short, the data is being captured from every event, action unlike the earlier manual system where most of the data used to be on the books. The volume of data is exploding everywhere and the time to process is reducing (actually the time is constant but the requirement for processing time is low). The data needs needs to be synchronized and made available to all the decision support systems in real time which means real-time data streaming is required to meet the business demand. The data could be residing on the cloud systems such as Salesforce.com or Microsoft Dynamics CRM cloud based systems which are hosted outside of the enterprise. For this to be efficient, this means that the data filtering and interchange should be minimal so that the data interchange time is used most efficiently. The demand for the enterprise IT integration software to select, filter, sort, compress, encrypt and process the data is having new challenges in this era. In simple terms, the integration software is expected to perform at a higher scale that it used to perform earlier.
