Open Source Business Intelligence: A Smart Move?

As business intelligence tools expand in cost and size, public CIOs test open source solutions for analytics.

by / December 21, 2008

Since 2001, the Wisconsin Department of Justice (DOJ) has been refining its computer-based data communications control center, which gives state law enforcement officials access to hundreds of data files at the national, state and local levels. The Transaction Information for the Management of Enforcement system, or eTIME, connects more than 9,000 criminal justice computers in Wisconsin and 500,000 across the United States and Canada from approximately 17,000 law enforcement agencies. Users can access files related to warrants, drivers' licenses, vehicle registration information, criminal histories, protection orders, injunction files, sex offender and corrections information, stolen property, missing persons and more.

With the vast amount of data eTIME processes and stores, Frank Ace, CIO of the Wisconsin DOJ, and Walt Neverman, the eTIME and technical services manager, embedded a reporting tool in the system. They also wanted statistical information on how the network was being used and data related to security and intrusion prevention, such as failed login information.

They needed business intellegence reporting capabilities - commonly referred to as BI - but didn't want to spend the money required for commercial business intelligence solutions from the likes of IBM and Oracle.

"Part of our challenge was to find a cost-effective way to produce reports for business managers of the system," Ace said. "In our line of business - where we collect information for the Wisconsin Crime Information Bureau - we needed to find a way to get information off systems in an effective manner."

Ace looked at a number of tools and weighed their cost and effectiveness before eventually turning to open source BI vendor JasperSoft for a lower-cost alternative to commercial BI software. Although he didn't expect the open source solution to be comprehensive and robust, he said the solution hasn't sacrificed quality.

JasperSoft's BI solution provides production reporting, interactive reporting, data analysis and data integration capabilities. The bureau has used JasperSoft's tool to provide reporting capabilities such as statistics on agency use, what transactions are being used most often, when the transactions are run, statistical information, security data and other patterns.

Ace said he hopes the detailed data reporting and analysis will provide a more effective system for law enforcement, improve service, determine maintenance needs, enhance security and reveal which agencies use eTIME the most.

The Intelligence Landscape
BI has consistently ranked a top priority for CIOs over the last few years, according to IT research firm Gartner. With the public sector storing record amounts of data, BI offers invaluable capabilities by collecting, analyzing and integrating data - and then creating detailed reports. BI provides historical, current and predictive views of business operations and supports the use of this information with data mining, analyzing, reporting and data integration features. Integrated reporting and analysis lets government managers better run an organization, improve services, identify effective strategies, enhance security and increase efficiency.

"Clearly business intelligence is one of the prioritized technologies within the government landscape today," said Rishi Sood, vice president of government consulting at Gartner. "Government CIOs believe new BI tools can help them become more efficient, improve service delivery and essentially do more with less. When looking at large-scale agency modernization plans, CIOs are demanding the insight from BI tools in order to improve performance and therefore lower cost of service delivery, or at least make service delivery more effective."

The major government segments using BI are: revenue and tax departments for modernization projects; administrative, finance, and budget offices for measuring projects' performance; and human services departments for tracking the effectiveness and progress of unemployment and social programs, Sood said.

At the federal level, numerous government agencies are using BI, including the U.S. Census Bureau, to analyze and predict demographic trends; federal health-care agencies to analyze and predict health statistics and fraud; the Labor Department to analyze economic data; the U.S. Securities and Exchange Commission to detect insider trading; and the IRS to detect filing discrepancies.

Large software firms have recognized the demand for BI and have been busy gobbling up small vendors. Examples include Oracle's purchase of Hyperion and Siebel Analytics, IBM's acquisition of Cognos, and Sun's purchase of MySQL. Other BI vendors include Microsoft, Hewlett-Packard, Business Objects and Unisys.

Analyzing BI's Value
Open source BI has emerged in recent years as an undercutting alternative to commercial BI solutions. By utilizing the low-cost, open source software (OSS) model that has gained traction worldwide among private- and public-sector agencies - including nearly 100 governments that have supporting pro-OSS legislation - open source BI has emerged as a viable alternative to its commercial counterpart. Vendors such as JasperSoft and Pentaho - two of the larger open source BI vendors - promise lower costs and the ability to access and personalize BI software code, which appeals to many private- and public-sector organizations.

"We have two trends playing in favor of open source BI," said Lance Walter, vice president of marketing for Pentaho. "One is the general purpose value of reporting and analysis to understand opportunities to save costs and improve efficiency. And the secondary piece is the fact that when public-sector organizations go with a commercial open source vendor, they tend to recognize significant savings over the up-front software license model offered by traditional providers."

Pentaho's services include dashboard, business framework, data mining, reporting and workflow capabilities. The public sector is the largest vertical industry in terms of adoption of Pentaho's services. The vendor boasts a public-sector client base that includes the Air Force, Food and Drug Administration, Department of Energy, Department of the Interior, Army, Department of Education and NASA.

The growth of open source BI in the public sector has been fueled partially by the size and expense of commercial BI software, according to Mark Madsen, president of Third Nature, a research and consulting company specializing in BI.

"BI vendors have thrown in so many features and modules that they are completely over-serving core needs and people are tired of paying six figures for something [of which] they use only 30 percent of the features," Madsen said. "Open source BI vendors are offering suitable products with a lower price for deployment."

The Ups and Downs of Open Source BI
According to a study conducted by North Bridge Venture Partners, the top factors that make OSS attractive are lower acquisition and maintenance costs, flexibility and access to source code, and freedom from vendor lock in. The same study notes that after the adoption of OSS, flexibility is seen as equally important as cost.

Open source tools are modular and designed for better integration in new architectures, Madsen said. Implementation of open source BI is often oriented toward developers who can take the software and embed data analysis into applications they write.

Although open source BI is technically free, Pentaho and other vendors offer subscriptions for their products that include access to support and services, such as updates, patches, future releases and help with code customization. While a subscription is often necessary for security and support, these costs are significantly less than commercial BI solutions. Another advantage to having a free initial download of open source BI is that public-sector organizations appreciate the flexibility and ability to test BI software before undergoing an RFP and purchasing subscription services.

Since open source BI solutions are relatively new, some government leaders have been hesitant to adopt them. The financial stability of open source vendors is questionable to some, with Pentaho and JasperSoft still receiving venture funding, which makes the longevity of future upgrades and next-generation releases unclear. Many government leaders seek vendors that have been established in the public sector, with solutions that are unique to them, Sood said.

"I think longevity, relationships and domain expertise are more important to governments than perhaps having the latest technology feature that no one else has," Sood said. "In a lot of respects, governments are interested in working with vendors they know are going to be there 15 years later."

Still a Niche Market
While European governments have embraced open source BI more so than North Americans - European governments are even promoting it - large-scale deployments are currently very rare. For now, open source BI has proven to be a valuable tool for selected functions in both the public and private sector. U.S. government clients have found niches for using open source BI in their service-oriented architectures.

"You don't have to use big commercial BI vendors and shell out $100,000 for a few servers and user licenses," said Andreas Bitterer, vice president of research for Gartner. "There are many application development projects that bundle JasperSoft reports because they want to have some reports based on whatever the BI application does, or a Pentaho suite, and it's great and works and performs reasonably well. But if I were the State Department or the CIA, would I rely on Pentaho or other open source companies? Absolutely not. It certainly depends on the scale, and we have not yet seen large-scale deployment for open source BI."


What's Your Priority?
Business intelligence was ranked as the No. 1 technology priority for CIOs in 2008 and has ranked in the top 10 for the third consecutive year.

Top 10 Business Priorities

  1. Business process improvement
  2. Attracting and retaining new customers
  3. Creating new products and services (innovation)
  4. Expanding into new markets or geographic areas
  5. Reducing enterprise costs
  6. Improving enterprise work force effectiveness
  7. Expanding current customer relationships
  8. Increasing the use of information and analytics
  9. Targeting customers and markets more effectively
  10. Acquiring new companies and capabilities (mergers and acquisitions)

Top 10 Technology Priorities

  1. Business intelligence applications
  2. Enterprise applications (ERP, CRM and others)
  3. Servers and storage technologies
  4. Legacy modernization, upgrade or enhancement
  5. Technical infrastructure
  6. Security technologies
  7. Networking, voice and data
  8. Collaboration technologies
  9. Document management
  10. Service-oriented architecture (SOA) and service-oriented business applications (SOBA)

Source: Gartner

Chandler Harris Contributing Writer