How do your technology services compare with others? Have you compared your service offerings, rates and customer satisfaction with similar size organizations in the public and private sectors? Should you be offering more for less? Elected leaders, CxOs and business managers worldwide constantly want answers to these questions. But benchmarking government technology services is easier said than done.

For one thing, every organization’s unique — with a different culture, local politics, history, budget, expectations and governance. Two, governments’ policies and requirements differ from the private sector’s — with Freedom of Information Act laws, diverse priorities from elected leaders, pay/benefit/cost-of-living disparities nationwide, and varying perspectives on outsourcing. Three, government service offerings must equally serve citizens and not “cherry pick” the most profitable services or segments of society. While value-for-money efficiency can — and should — be measured in government operations, the private sector differs.

This makes us realize that benchmarking isn’t an exact science. Developing actionable measures requires excellent methodology, a large database of comparable entities, good judgment and lots of experience. During one benchmarking effort a few years ago, one person said, “If this is apples to apples, one is a Golden Delicious and the other is a Granny Smith.”

Nevertheless, comparing costs and service levels is a pragmatic strategy that’s an essential element of effective leadership. Most government technology managers that I’ve met believe that their teams are providing good value for the money spent given their set of circumstances. Still, they recognize the need for external validation and want to benchmark. Managers who resist benchmarking are sometimes pressured by new business leaders to bring in the “experts from out of town” to determine if changes can save dollars or offer better services.

Michigan recently completed a benchmarking effort of technology infrastructure services. It was a bumpy multimonth process — and we almost drove off the road a few times. The journey was painful, but worth it. I’d like to share several things we learned and offer a few (vendor-agnostic) tips to help in your benchmarking.

  • How we got started — Michigan’s been working on a potential new data center project for more than a year. After the Request for Information phase, we learned that we needed much greater detail before moving to the Request for Qualifications (RFQ) phase. Our benchmarking effort was critical in preparing for the RFQ and RFP phases. Tip: Consider linking benchmarking efforts to a wider initiative with executive support.
  • Initial team and plans — After awarding the contract competitively to a company with impressive benchmarking expertise, we built an internal team with staff from many different areas. Tip: Ensure all stakeholders are included in the process.
  • Different definitions — After a successful kickoff, our joint teams met many times to fill out spreadsheets and describe our rates and services to the consulting experts. We were compared to public- and private-sector organizations of similar size and complexity. Tip: Have clear definitions, especially around the scope of what’s included in all metrics.
  • Troubles emerge — At one point, we had a major disagreement with our vendor partner. Early results showed that our costs were well above others. But further analysis revealed that we weren’t comparing apples to apples, e.g., we included much more consulting, research, development, testing and evaluations within our rates. Tip: Do a comprehensive “FTE-mapping” of how each person spends his time. Stick with the program, even if you initially disagree with the numbers.
  • Back on course — Later, a top executive from the consulting company said, “This often happens. Customers either think they are too low or too high in some area. It’s a part of the benchmarking process.” In the end, we found the results helpful with our technology services and rates ranging from best practices to areas requiring improvement.
  • Final thought: Benchmarking is a must. Lord Kelvin once said, “If you cannot measure it, you cannot improve it.”

Dan Lohrmann is Michigan’s CTO and was the state’s first chief information security officer. He has 25 years of worldwide security experience, and has won numerous awards for his leadership in the information security field.

Dan Lohrmann Dan Lohrmann  |  Contributing Writer

Daniel J. Lohrmann became Michigan's first chief security officer (CSO) and deputy director for cybersecurity and infrastructure protection in October 2011. Lohrmann is leading Michigan's development and implementation of a comprehensive security strategy for all of the state’s resources and infrastructure. His organization is providing Michigan with a single entity charged with the oversight of risk management and security issues associated with Michigan assets, property, systems and networks.

Lohrmann is a globally recognized author and blogger on technology and security topics. His keynote speeches have been heard at worldwide events, such as GovTech in South Africa, IDC Security Roadshow in Moscow, and the RSA Conference in San Francisco. He has been honored with numerous cybersecurity and technology leadership awards, including “CSO of the Year” by SC Magazine and “Public Official of the Year” by Governing magazine.

His Michigan government security team’s mission is to:

  • establish Michigan as a global leader in cyberawareness, training and citizen safety;
  • provide state agencies and their employees with a single entity charged with the oversight of risk management and security issues associated with state of Michigan assets, property, systems and networks;
  • develop and implement a comprehensive security strategy (Michigan Cyber Initiative) for all Michigan resources and infrastructure;
  • improve efficiency within the state’s Department of Technology, Management and Budget; and
  • provide combined focus on emergency management efforts.

He currently represents the National Association of State Chief Information Officers (NASCIO) on the IT Government Coordinating Council that’s led by the U.S. Department of Homeland Security. He also serves as an adviser on TechAmerica's Cloud Commission and the Global Cyber Roundtable.

From January 2009 until October 2011, Lohrmann served as Michigan's chief technology officer and director of infrastructure services administration. He led more than 750 technology staff and contractors in administering functions, such as technical architecture, project management, data center operations, systems integration, customer service (call) center support, PC and server administration, office automation and field services support.

Under Lohrmann’s leadership, Michigan established the award-winning Mi-Cloud data storage and hosting service, and his infrastructure team was recognized by NASCIO and others for best practices and for leading state and local governments in effective technology service delivery.

Earlier in his career, Lohrmann served as the state of Michigan's first chief information security officer (CISO) from May 2002 until January 2009. He directed Michigan's award-winning Office of Enterprise Security for almost seven years.

Lohrmann's first book, Virtual Integrity: Faithfully Navigating the Brave New Web, was published in November 2008.  Lohrmann was also the chairman of the board for 2008-2009 and past president (2006-2007) of the Michigan InfraGard Member's Alliance.

Prior to becoming Michigan's CISO, Lohrmann served as the senior technology executive for e-Michigan, where he published an award-winning academic paper titled The Story — Reinventing State Government Online. He also served as director of IT and CIO for the Michigan Department of Management and Budget in the late 1990s.

Lohrmann has more than 26 years of experience in the computer industry, beginning his career with the National Security Agency. He worked for three years in England as a senior network engineer for Lockheed Martin (formerly Loral Aerospace) and for four years as a technical director for ManTech International in a U.S./UK military facility.

Lohrmann is a distinguished guest lecturer for Norwich University in the field of information assurance. He also has been a keynote speaker at IT events around the world, including numerous SecureWorld and ITEC conferences in addition to online webinars and podcasts. He has been featured in numerous daily newspapers, radio programs and magazines. Lohrmann writes a bimonthly column for Public CIO magazine on cybersecurity. He's published articles on security, technology management, cross-boundary integration, building e-government applications, cloud computing, virtualization and securing portals.

He holds a master’s degree in computer science from Johns Hopkins University in Baltimore and a bachelor’s degree in computer science from Valparaiso University in Indiana.

NOTE: The columns here are Dan Lohrmann's own views. The opinions expressed do not necessarily represent the state of Michigan's official positions.

Recent Awards:
2011 Technology Leadership Award: InfoWorld
Premier 100 IT Leader for 2010: Computerworld magazine
2009 Top Doers, Dreamers and Drivers: Government Technology magazine
Public Official of the Year: Governing magazine — November 2008
CSO of the Year: SC Magazine — April 2008
Top 25 in Security Industry: Security magazine — December 2007
Compass Award: CSO Magazine — March 2007
Information Security Executive of the Year: Central Award 2006