Understanding the Impact of New Technologies on BI

"Rather than relying solely on a rigid metaphor like data warehousing, BI needs the ability to access data anywhere it can be found and to perform integration on the fly, if necessary. Locating the right information to solve problems must be a semantic process, not requiring knowledge of data structures or canonical forms."

Neil Raden, Intelligent Enterprise, Business Intelligence 2.0

A long-standing concern between business teams and IT is the question "Have we spent the past years creating some monolithic solution that is now unmanageable?" Can it come close to addressing the needs of the current and future business? Have the Business Intelligence (BI) systems that we've created become rigid metaphors, as Neil Raden proposes that data warehouses have become?

Implementing a BI system has always been hard and costly. But the value, once up and running, can be enormous. Insight isn't something to be taken lightly; it can change the very nature of how a business is run. It is no wonder, then, that the markets for BI and performance management solutions are expanding rapidly.

The business needs for BI are going far beyond traditional analysis and reporting; companies are now expecting information access in real-time, across global constituencies, and beyond the limitations of financial data, or for that matter, data itself. The BI systems that we built over the past decade simply may not scale to meet today's requirements.

New technologies such as software as a service (SaaS) are impacting the very nature of software development and delivery, and are also helping to shape the future direction of BI. Even large vendors such as SAP and Oracle are testing out the on-demand waters, SAP with its' 'Business by Design' on-demand solution, and Oracle delivering Oracle Hyperion On Demand. While SaaS has yet to experience broad acceptance in the marketplace, niche BI companies such as LucidEra, Adaptive Planning, Host Analytics and PivotLink are gaining customers with their on-demand offerings, and open the market to small and mid-size (SMB) companies.

Open source BI offerings may provide a clue into another alternative direction for BI markets – again with potential business benefit. As the time and cost of implementing and supporting BI solutions are reduced, so too will the bar lower to a level where SMBs can take advantage of BI, while larger companies can customize solutions and extend analytics across the enterprise. BI will broaden from the hands of the few into the hands of many.

While the new technologies broaden the BI market offerings, they exacerbate the issues companies currently face managing financial data across increasingly stratified and expanding markets. We are already dealing with a crisis in data management. Critical financial information is stored in proprietary BI systems and cannot be easily extracted or combined with operational data. Information from all source systems must be auditable and compliant. It must exist in standard structures that will support future development of combined data with other content sources. Before we can truly scale to meet the coming needs of the BI market we must create a reliable and consistent method for moving and managing data across the enterprise.

The increasingly complex business demands on BI systems are a further validation of the success and relevance of the market. We can learn from other rapidly-evolving technologies that have gone through major expansions as a result of their cumulative relevance. The Web, and Web based standards come to mind as an example of a technology that has successfully transitioned into a much higher form of relevance. Standardization, interoperability, and security are core Web concepts. Before we can hope to truly leverage new technologies, we must overcome the limitations that we've built into our existing BI systems, ensuring standardized data access mechanisms, real-time, secure access to data, and a framework that will extend into the next realm of BI technology development, and Radan's vision of "Locating the right information to solve problems must be a semantic process, not requiring knowledge of data structures or canonical forms."