Implement a data integration solution in six easy steps
This is the first installment of a two-part series expert-advice on data integration solution deployment.
This is the first part of a two-part series on BI data integration. See the second part for integration challenges.
Today, data integration solutions are deployed to support a plethora of business initiatives and technology implementations. It is important for organizations to understand how to go about implementing these solutions, if they are to avoid problems later. This first part of a two-part series looks at the key considerations to keep in mind when deploying a data integration solution.
The most important consideration when implementing a data integration solution is to ascertain whether there is a strong business need for it. Here are three business scenarios which may call for it:
- One-view of data across the group or business: An example would be the situation following a corporate merger or acquisition, when you need to integrate all the enterprise data. At Hypercity, we have multiple group companies specializing in retail: Hypercity, Shoppers Stop, and Crosswords. We needed to integrate customer data across all these businesses to serve our customers better.
- Facilitating data flow across systems: There may be situations where you need to integrate multiple data sources and applications to execute a business process; for example our business analytics tool, where the data flows from multiple applications like the merchandise management system or Oracle financials.
- Enabling data integration when new applications are deployed: A new enterprise application will need data from all the existing applications. At Hypercity, when we implemented our home delivery application, it needed to source customer- and product-information from the existing systems, making data integration a critical business need.
Move directly to Data integration in parts >>
Key steps involved
- Work with the software vendor
The first step in deploying a data integration solution is to collaborate with the software vendor. It is in your interest to help the vendor understand your business requirements correctly. This will enable the vendor executives to accurately identify and integrate all the required data points.
- Define and document integration priorities
List down all the integration tasks required. For instance, if you are deploying a new application, study its deployment schedule. Your aim should be to complete all data integration activity prior to deployment or Go-Live stage. Also, document the frequency at which incremental data should be updated. In addition, assess and document the benefits due to data integration solution, in terms of savings in cost and time.
- Identify the appropriate integration interface
Data integration solutions provide two kinds of data interfaces: one-way and two-way. Know where to apply which one.
In a one-way interface, data is sent from point A to point B only; there is no return or backward movement. On our B2B portal, our suppliers can track stock movement in the stores. The inventory, payment, and sales information are sent to the B2B portal, but no data is sent back to these data sources.
In a two-way interface, data is sent from one application to another and back. In our case, when a new application such as point of sales (POS) is deployed, the product data is sent from the merchandise management system to POS, and then sales data is sent back again from the POS.
- Select the right interface medium, may-be not the easiest one
In choosing the right medium to deploy a data integration solution, consider future needs as well as upgrades. Data integration methods include XML, comma-delimited, spreadsheet, direct database connection, and so on. The best method for you may not be the easiest one. Base your decision on scalability requirements, volume of data, and cost of the solution. Text-based integration had been prevalent for years, but XML and direct database connection are the popular modes employed today.
At HyperCity, during our integration of merchandise management systems (MMS) and point of sale (POS) solution, we noticed there was product information being generated by MMS in text format. But the POS solution required it in HTML format. We initially thought the easiest option was to convert the text to HTML. However, the conversion ended up taking longer than desired. We then realized that the better option was to create HTML files at the source system itself.
- Monitor the process, put various check points
Fine-tune the data that is being integrated. For our B2B portal, data integration occurs at the end of each day, and only the incremental data is pushed to the server. At Hypercity, there are check points at every level, starting with the source system. Sales questions such as “How many quantities were sold?” and “What were the retail prices?” are raised at these check points. The extract and load stages of the B2B portal form the second check point, and the same questions have to be answered here as well. Automated systems check the accuracy of the entered data. In case of error, an email is sent to the concerned user that the data for the day is not uploaded properly, and corrective measures need to be taken.
Your data integration checkpoint design should not consider the current business-needs only as these investments rarely bear fruit in the short term. A three- to five-year information management roadmap for the organization should be factored in when creating the design.
- Ensure security of the data
Set security policies according to risk level. If you plan to transfer the data on the internal network encryption may not be required. But if you need to send it over the Internet, it may be needed to prevent tampering.
Data integration in partsData integration can be divided into three different parts, based on associated initiatives.
|
About the authors:
Dipthi Karnad is a senior IT Manager at HyperCITY Retail India. She has over 16 years of experience in IT, of which 13 years are in the Retail IT domain. She has an in-depth understanding of retail business processes and expertise in designing and implementing solutions across platforms. Dipti holds a masters degree in financial management.
Kapil Tulsan works as an IT team lead at HyperCITY Retail India, and has over ten years of IT experience. He has been involved in system requirements study, analysis, design, development, and deployment. He has worked on several projects with multinational companies.
(As told to Mitchelle R Jansen)
