Five years ago, Web application development in North Carolina was a hit-or-miss operation. Some state programmers failed to meet testing standards, and some didn't test at all. North Carolina needed to overhaul its application-testing processes, preferably without spending too much cash to make it happen.
In 2004, the North Carolina Office of Information Technology Services (ITS) realized it had to change how state agencies performed software quality assurance (SQA) testing on their applications, found programming faults and created criteria for developers.
"Depending on which agency you walked into, you could see some good [software development life cycle] methods and techniques being used and the appropriate tooling and so forth available for doing quality assurance," said Gary Alexander, ITS director of service portfolio management. "Then you could walk into some other places and literally, it was nowhere to be found. And there were obviously flavors of it in the middle too."
He came to state government in 2003 and saw that the lay of the IT land was rather rough, so he wanted to do some landscaping. Specifically, he wanted to expand the number of services ITS delivered to agencies.
Fortunately the time was right to act: In 2004, the state passed SB 991, which mandated IT infrastructure consolidation. Alexander, along with ITS colleagues, created a concept for expanding and modifying the office's service delivery processes and garnered approval from CIO George Bakolia.
The office decided to save North Carolina money and headaches by consolidating SQA testing methods under ITS. The plan was to allow agencies to subscribe to quality assurance tools provided centrally by the ITS instead of purchasing tools themselves. This would unify and streamline application testing with less fuss.
"Every day we're using the service and testing applications for various state agencies that had been going untested and would have been launched with some very painful and obvious bugs or performance issues associated with them," Alexander said.
The office provides agencies the SQA tools for a fee through a Web-based delivery model known as software as a service (SaaS). It's a popular way for CIOs to subscribe to software and use it over the Web, and often it's a cheaper alternative to buying the software and hardware to run it. In North Carolina's case, the SQA software is owned by ITS, and state agencies rent it to test their own applications.
"You have a single incidence of software, hardware and network infrastructure that is supporting and servicing multiple clients or tenants, so it's the one-to-many type of scheme," Alexander said.
SaaS Usage Grows Significantly
The SaaS delivery model has been quickly spreading globally, according to prominent business research. Gartner, an IT research and advisory firm, reported in March 2007 that the worldwide SaaS market reached $6.3 billion in 2006 and forecast that it would grow to $19.3 billion by the end of 2011. In January, IT research company IDC estimated that 76 percent of American organizations would use at least one SaaS-delivered application by the end of 2009.
Arguably the main reason these organizations try SaaS is the upfront cost savings on implementation. For starters, the software runs on servers owned by the SaaS vendor. In most cases, the host also mobilizes its own IT personnel to handle problems or system upgrades, which means IT departments don't have to worry about maintenance issues. Customers pay for usage, not ownership.
"The other benefit by consolidating all the licenses and offering the tools under one agency is that we can control all the patching, upgrades and support in that nature," said Paul Schmidt, service owner of Software Quality Assurance Services at ITS. "If we had it spread on all agencies, each one of them would have to have an administrator do all that, and it gets a little cumbersome in that respect."
Photo: The Software Quality Assurance Services team makes sure state government has the tools it needs to test Web applications. (Front row, left to right: SQA Specialists Laura Coyle, Clay Adams and Angela Langley-Murphy; Back row, left to right: Paul Schmidt, service owner of SQA Services and Gary Alexander, director of Service Portfolio Management.)
According to ITS, North Carolina's agencies can access software testing tools 24/7 and 365 days a year, excluding planned outages, maintenance windows and unavoidable events. ITS will also handle agencies' customer support requests regarding the service. Schmidt has a team of three who handle administration duties.
After the office devised the plan in 2004 to offer SQA services in a SaaS delivery model, ITS solicited proposals from vendors and selected technology from Mercury Interactive Corp., an organization now owned by Hewlett-Packard. ITS purchased the technology, installed it on state hardware and used it to host SQA tools for paying agencies. In June 2006, the office made a final proposal to executive management about how the service would be performed, and on August 2007, the service was launched with eight agencies onboard as customers.
These tools have come in handy at the Employment Security Commission (ESC), an agency offering job search databases and unemployment benefits information.
"One of the things that we're very concerned about, since most of our applications are public-facing applications, is that they perform as we would expect them to before they're released to the public," said Robert Cottrell Jr., the commission's information systems director.
Robert Cottrell Jr., information systems director, Employment Security Commission
Cottrell's agency provides Web applications for various groups. Services for citizens include filing for unemployment benefits, searching for jobs and estimating the types of benefits based on wages. Business services include filing various types of worker claims, searching electronic job-applicant records and filing waste records. And unlike most other state agencies, ESC's digital jurisdiction stretches beyond North Carolina -- people in other states can use ESC's applications for job searches and other functions.
Alexander said agencies' ability to catch mistakes in their applications before public release can save them from embarrassment.
"When you push a Web site off to the public, and some elected official is standing on a podium in front of the press spewing out information about what this Web site will be able to do or what you get from it, you could have thousands of people attacking that Web site within minutes of that press announcement," he said.
Thousands Spent, Millions Saved
ITS currently offers SQA at three service levels. The Tier 1 service level, which is $8,900 a month, allows functional testing -- from a user's perspective -- for 10 to 20 applications monthly and performance testing -- from a developer's perspective -- for one to five applications monthly. Tiers 2 and 3, which cost $6,300 and $4,200 a month, respectively, provide progressively lower volumes of functional testing capability. Agencies that subscribe can also pay a separate rate, if they wish, for guidance and support of ITS subject-matter experts.
North Carolina skyline. Photo copyright by iStockphoto
"Before the service, they would have to go out and secure a contractor and then maybe get somebody to have the right skill set," Schmidt said of agencies' ability to acquire qualified testing personnel. "Where the difference is now, they can call upon us and say, 'Hey, we need this resource in this amount of time.' They don't have to go through a procurement process to do that."
The service earned a noteworthy benchmark of success when North Carolina became a finalist in the 2008 National Association of State Chief Information Officers Recognition Award category for IT project and portfolio management.
The service has saved the state millions of dollars. According to the award program submission, the state saved an estimated $30.6 million in implementation costs. ITS paid $4.6 million in enterprise costs for the SaaS system compared to the $35.2 million the other eight participating agencies would have paid if they purchased their own SQA testing software. The figures were derived from vendor price lists and were based on estimated five-year cycle costs.
Perhaps the strongest sign of success for North Carolina's SaaS initiative is the consistent demand from participating agencies for the software testing system. Since the service's 2007 launch with eight participating agencies onboard, that number has grown to 10 as of 2009, and more have expressed interest.