About a year ago, the term "enterprise service bus" (ESB) suddenly appeared in IT publications. The first wave of stories examined the emerging technology, which uses a messaging platform as a middleman between systems and applications. Later, articles described how early private-sector adopters capitalized on ESB's potential to drastically simplify integrating the usual lot of discrete systems producing their own data or doing their own thing in the overall enterprise.
Overlooked was an early adopter in the public sector -- Wisconsin took to the technology as quickly as its commercial contemporaries. The state's Division of Enterprise Technology (DET) became interested in ESB technology in May 2003, finalized procurement in December and deployed an ESB in February 2004 -- a fairly aggressive timeframe for government.
"Screw those people who said we couldn't be cutting edge," said Wisconsin CIO Matt Miszewski. "If there's a specifically identified business need, we try to address it, if we can, with an IT solution. That's exactly what we did here. We didn't care how long it's been around and whether the private sector had adopted it and whether the public sector was on board.
"We've found out since then that we're well ahead of the curve," he said. "Several large ISVs [independent software vendors] have come to find out what we're doing to see if they need to have product offerings that support bus solutions. That shows we're a little ahead of both the public and private sectors."
Miszewski estimates that ESB technology -- which uses XML and Web services to simplify the task of exchanging information among incompatible applications and hardware platforms -- saved his state millions over competing enterprise integration strategies.
Since initial deployment, the DET started six other initiatives within the Department of Administration (DOA), said Werner Gade, section chief of Enterprise Infrastructure.
Gade said the DET, a division of the DOA, is working with the state court system to allow the DOA to gather statistical information from Circuit Courts for reporting to the federal government, the Office of Justice Assistance on a statewide extension of the ESB to 610 law enforcement agencies, and the Department of Revenue on the early stages of a tax-auditing project, not to mention integrating the DOA's purchasing system with its accounting system.
In the past, middleware solutions integrated specific applications, forcing agencies to laboriously map data fields back and forth between the applications. These point-to-point solutions are custom tailored for each situation, so they have limited benefit beyond the particular task for which they were designed. Therefore, agencies face a new integration project each time they want to connect dissimilar systems.
The ESB brings a new premise to integration: Why not use XML messaging technology to route data among applications? XML messaging technology allows data sending between applications and platforms the way e-mails are sent between end-users.
Using messaging technology wasn't an option a few years back -- when Web services were wobbling around on unsteady legs, standards were weak and ESB technology itself wasn't sufficiently mature.
Advances in asynchronous messaging technology mean the backbone, instead of the systems and applications, can now do the real integration work -- transforming data into a form capable of being shared by disparate platforms, formatting data for sending or receiving and orchestrating data delivery to a wide range of applications in an enterprise's different systems.
According to Dennis Byron, research analyst for application deployment software at IDC, an ESB makes the most of services created by a service-oriented architecture (SOA) -- a way of building IT infrastructure many enterprises are working toward. The basic premise is that a "service" is a function performed by an application at an end-user's request.
For a practical example of an SOA, think of watching a DVD. The DVD player provides a service -- playing the DVD -- that works with any DVD. So you can pop a Sopranos disk in any player and it works. In a point-to-point architecture, every Sopranos DVD would have to be watched on a specific DVD player. The DVD is tied to that particular player, and must not be separated.
In an SOA, applications are broken down into services, Byron said. "The logic is somewhere, and the data-access code is somewhere else," he said. "The part of the application that says 'print the tax bills' is a service."
Even 30 years ago, monolithic mainframe applications were written this way, but usually linked only to other COBOL programs running on the same hardware. An ESB takes that concept and makes it work across dissimilar applications and hardware platforms.
The ESB architecture is somewhat similar to its integration predecessors, but an ESB relies extensively on storing and forwarding service messages.
"A service raises its hand and says it needs something done, and another service is typically sitting there saying, 'I can do that,'" Byron said. "Or a service may be sitting there ahead of time saying, 'Whenever anybody wants this done, I can do it.'"
Breaking applications down into services allows those services to be reused in other ways. "Decompose what you're trying to do into as many discrete activities as possible," he said. "You might find that printing the tax bills is not a lot different than printing water utility bills, so you decompose it in such a way that a lot of the same code can be used for both of those two functions."
This reuse of code is one thing that attracted Wisconsin, Mizsewski said, and agencies play an important role in creating code that can be shared across the enterprise.
"Part of the benefit we get out of the ESB is that application development, including Web services development, can reside in the agencies where the expertise should be," Miszewski said. "They know the business needs of their individual departments. They should be developing the Web services we have.
"Agencies should be developing Web services with an eye toward reuse -- and reuse for the entire enterprise -- so we start to develop a large library of reusable code that can easily and quickly be deployed from the bus or in other ways," Miszewski said.
Catching the Bus
"In terms of technologies we roll out, the need for this is pretty obvious," Miszewski said, noting that the DET has been inundated with agency requests to try the ESB on for size since the initial test run. "Much like other states and other large companies, we have a relatively largely decentralized IT environment that's extremely heterogeneous."
The DET is using the ESB architecture to help agencies share information despite the decentralized IT environment in which they operate. Previously the agencies tried to integrate certain systems by creating point-to-point solutions for specific circumstances, Miszewski said.
"As the need arose, they would code a solution that would connect two systems or three systems together," he said. "Then when the need arose again they would connect maybe the same system with a different system -- as opposed to doing it the right way, which is by utilizing an information or services bus."
Wisconsin wanted to make agencies aware of enterprise-driven integration and get away from project-driven integration because the proliferation of ad hoc point-to-point solutions created by agencies was costly.
"As a part of any number of projects, developers would need to have access to remote data, and each of them would head off for their own solution based on their experience and the tools they had at their disposal," said Erik Mickelson, an enterprise technical services specialist with the DET, who first proposed an ESB to Miszewski.
"That meant the state was paying for the same problem to be solved each time somebody faced it," Mickelson said. "We're just recognizing that integration is a problem we all share and all have an interest in solving together in as simple and effective way as possible."
The DET is offering agencies a set of templates or patterns to use in hooking themselves to the ESB. Packages of adaptor technology are another set of tools being extended to agencies to help them connect Web services to their legacy mainframes.
"When we first started this initiative, we didn't really know what to expect in terms of what my group [Enterprise Infrastructure] was going to be," said Gade. "We came to realize that the technology behind the ESB is the easy part. Our role is becoming more of working with the business areas in helping them understand how they can use this technology to their benefit. We recognized that we needed an 'integration competency center' that is meant to help business people understand how to use the tools available to them to integrate with other components of the enterprise."
The New Acronym
ESBs dramatically reduce the cost of integrating enterprise applications, according to Miszewski.
Wisconsin spent $300,000 on an ESB solution from Cape Clear Software Inc., instead of millions on enterprise application integration (EAI) software, he said. The difference was that Cape Clear's offering was an ESB out of the box.
"The other EAI/ESB solutions became expensive because to achieve Cape Clear functionality, we were required to buy, install, deploy and operate most of the large players 'components' simply to piece together a bus," Miszewski said.
A host of vendors now offer ESB products, including Cape Clear, Fiorano Software, Software AG, Sonic Software and others. Even the big boys are getting into the act -- IBM and Microsoft are working up their own branded ESB products, but there's no firm date when those ESBs will hit the market.
The technology clearly is hot, though the first products labeled as ESBs hit the market in early 2002, and opinions vary as to why two years elapsed before enterprises of all sorts jumped on the ESB architecture.
Miszewski said part of the problem is that early ESB solutions didn't emphasize what clients wanted -- cost savings. Enterprises weren't going to spend $3 million to $5 million on an integration solution that costs nearly as much to deploy, he said.
"When the pure plays came out, Sonic Software, Cape Clear and others came out with a price point that made sense," he said. "If you're going to try a new technology that's standards-based, it's much easier to try it with some of the companies we talked to than some of the large companies.
"Also, my counterparts, having come out of the dot-com bursting and a lot of bankruptcies that interrupted their operations, are leery of doing business with smaller, more aggressive companies," Miszewski said.
To others, the ESB is right on its evolutionary track.
ESB is like HTML, said William Ruh, CTO and senior vice president of Software AG, citing HTML's ascendance over five years from a "What is this?" technology to "Everybody just does it" technology.
"I think we're in the first year of that five-year cycle," Ruh said. "If you remember in 1996/97 you sort of heard about HTML and then bam! It just sort of blew open. My personal feeling is that we're going to see the same thing happen with Web services and ESB within the next year."
The technology's relative newness is perhaps another reason it hasn't been fully accepted in enterprise architectures, though observers, including Gartner and IDC, predict the ESB will take on a central role in enterprises by 2005.
Ruh said Software AG has approximately 70 clients using its ESB product, Service Integrator, and has worked with two states -- South Carolina and Massachusetts -- to deploy ESB technology. South Carolina's public employees retirement system is using the software, as is Massachusetts' Division of Professional Licensure.
Government enterprises of all sizes should take a serious look at the technology, Ruh said, because of its flexibility and affordability.
"The ESB does lend itself to small organizations, as well as big ones," he said. "This is what makes it so different from earlier, proprietary technologies and other heavyweight standards, like CORBA or DCE. These things were meant to be used by very large organizations, and both ESB's price point and utility are attractive to small and medium-sized government organizations.
"ESBs in the market today are between $5,000 and $25,000, depending on the functionality we have," he said. "That's approachable by any organization out there."
Sonic Software also has pursued the ESB market, said Hub Vandervoort, vice president of strategic services, and the company is happy with its ESB product's performance.
"At this point, we're a little over two years in market with this product," Vandervoort said. "We're at about 180 named customers that have licensed the ESB. Well over half of those are in production, and in some cases, are in their second or third generation."
He said he sees ESB technology as the next iteration of integration technology because an ESB takes integration to the enterprise level. EAI technologies were sufficient for integration within a department, he said, or LAN-oriented integrations of up to five applications.
Expanding this sort of integration to an enterprise level foundered due to EAI's reliance on hub-and-spoke architecture, where everything runs through one processor and is sent out to PCs connected to that processor.
"That's fine, to a point," Vandervoort said. "Metaphorically can you imagine if the whole Internet ran through St. Louis? It would be a big scalability bottleneck, wouldn't it? It wouldn't work for a global type of usage."
The first generation of EAI software hit the wall in late 2001 and throughout 2002 because the software's cost for the yield just didn't add up for organizations, he said, another reason the ESB architecture took on added luster.
"By design, because of its distributed architecture, an ESB doesn't run in one place or one machine," Vandervoort said. "It runs everywhere. A little bit of it runs on every machine that participates in the integration network. Because it's distributed, you can buy it in bite-sized chunks, and your cost of entry is extraordinarily low relative to EAI."
In addition to the low cost of entry, the ESB gives Wisconsin the chance to make the most of its decentralized IT environment, perhaps the most attractive draw of the ESB approach, said CIO Miszewski.
"You're able to actually leverage the heterogeneous systems out there at the same time you share both data, and potentially, pieces of those individual applications," Miszewski said. "Instead of fighting the fight -- there's different database systems, different operating systems, different coding languages out there -- you can avoid that and leverage it all into one system with the bus."