Build open source business intelligence platform successfully

A business intelligence platform binds the BI application layers and enables improved integration, information delivery and analysis. While proprietary vendors sell BI systems as fragmented products, buying all the pieces may prove to be overly expensive. On the other hand, organizations may procure an all-in-one package only to discover that one size does not fit all.

Building your own business intelligence platform offers a good alternative. A business intelligence platform is a technology-framework that facilitates development of innovative business intelligence solutions. (See Box: What is a business intelligence platform). An open source business intelligence platform offers greater flexibility and scalability than branded options.

Key ingredients

Any recipe for an open source business intelligence platform calls for a dash of BI experience and proficiency. A single open source framework or model to stage and integrate data may not satisfy all platforming needs. Thus, knowledge of the options available is important to develop a cohesive solution. What an open source business intelligence platform does is help fabricate a standardized system of functioning at an affordable cost.

Evaluation checklist

Use the following checklist to evaluate your open source business intelligence platform:

• What level of stability will the tool provide, in terms of integration and obtaining single version of the truth?
• What problems would the business intelligence platform address?
• How flexible will configuration of the tool be?
• What is the extent of community contribution and potential growth of the technology?
• What are the implications of choosing that tool?
• What security environment does the framework conform to and allow?

The nitty-gritty

From a developer’s point of view, some training and experience is necessary to develop an open source business intelligence platform. These resources may not be available as easily as they are in the proprietary world. Common frameworks such as Tao have well-defined training modules and trained professionals available; also, developing Java-based platforms is now the rage. The core open source BI framework chosen is important. One needs to consider which add-on framework will be used to build a larger platform, and whether the BI technologies already available will work with the platform as a whole.

For specialized reporting needs, customizations would be required. As the business intelligence platform is open source, it would be easy to modify the code to incorporate these needs. It is here that integration partners come into the picture. Meanwhile, the organization’s developer team gets involved and trains the users on the nuances of the business intelligence platform, enabling them to provide feedback, which can be used by the team to further advance the solution.

Infrastructure considerations

Besides the source data, the prevalent IT infrastructure must be considered before setting out on the open source business intelligence platform journey: 

• The choice of technology should be in line and work well with your open source business intelligence framework. Also, consider the technology already in place before selecting anything new.
• Assess future plans to host your BI system on the cloud.
• Ascertain hardware and network environment needs for the business intelligence platform to succeed.
• Find out how the platform will interact with multiple data sources and how information is injected into it.

For the infrastructure, there are several models one may choose from. A simple cloud model is an available option. For instance, using the services of a large cloud vendor different instances of the business intelligence platform may be developed for tests. Assimilation of information from multiple data sources and systems is the norm. This requires a robust data integration process, and ETL plays a major role here. Some platforms would need an ETL integration layer written into the platform, which hooks onto clickstream data from other systems. This would have to be built in a multi-stage process, wherein different levels of data, depending on their sanctity, can enter the platform to be utilized by the BI reporting tools.Figure: Business intelligence platform illustrated

Data quality

In most cases some sort of data treatment may be prevalent. However, for new initiatives, a quality assessment would have to be conducted and quality control undertaken at the data generation level. There are tools available for this. Ideally, when the ETL is being modified to include the business intelligence platform, quality checks must be installed. A rework of some of the data sources might be required. A threshold for quality would have to be established as the tolerance level, depending on the business processes.

About the authors:

Shubham Nagar is the CEO of InfoAxon Technologies, a provider of open source based enterprise services and solutions. He is responsible for InfoAxon's growth, strategy, and identifying new business opportunities for InfoAxon. Vineet Dahiya works as director – business development at InfoAxon and is responsible for new business and revenue generation for the company. Nikhil Kapoor is the CTO of InfoAxon and leads the company’s BI practice.

(As told to Sharon D'souza)