When we noticed that less than 50 percent of its data backups succeeded on the first pass with a low chance of successful restores, we realized that cost savings were secondary only to performance. These days, I sleep better at night because of effort put forth in streamlining the company’s data management.
When it comes to data management, organizations within the utility industry are doing a little bit of everything, but mostly, they run operation technology (OT) and information technology (IT) as completely separate islands. This is not the case at AMP, where the data and technology management of (most) OT and (all) IT resources have been centralized to drive business efficiency.
This is not a new concept; however, people are still quite startled when we talk with others in the industry about what we’re doing. At AMP, we may be pushing the data management envelope by streamlining our technology departments, though we are actually on the cusp of a growing trend of bridging the divide between OT and IT that has been seen in many sectors recently.
OT has become increasingly more like IT over the years. The functions each department plays are different, but the core technology and infrastructure are often the same. Having separate and distinct OT and IT roadmaps often results in duplicate efforts and costs, as well as creates a huge liability and security risk. For instance, many OT systems and processes now utilize traditional TCP/IP networks and industry standard operating systems (Windows/Linux). When the two separate departments are both worrying about things like cybersecurity, configuration and change control, you’ll have one department doing X while the other is doing Y, which is not only counterintuitive but also a potential waste of time, resources and money.
Alternatively, if both departments are connected and have insight into what the other is doing, your organization as a whole is able to move forward in a more efficient way, which is what we have been able to achieve by leveraging key technologies, such as CommVault’s Simpana software. With one platform that crosses and helps manage both IT and OT departments, we’ve been able to eliminate many custom built applications. By standardizing on Simpana’s software management layer to control both sets of data through a single interface, AMP has:
- Consolidated 10+ backup systems/processes into one single platform for global governance across a fleet of sites;
- Integrated and consolidated many IT and OT systems for centralized management and reporting, helping to predict growth and reduce hardware/software spend across the enterprise;
- Virtualized and created a secure and non-disruptive backup and restore process for many SCADA and plant systems;
- Replaced a highly customized and unsupportable SCADA system with a market leading system that was completely virtualized and vendor supported with reduced customization;
- Introduced automation for the deployment and detection of new servers and systems (virtual and physical), enabling quick protection of new environments;
- Improved backup and restore success rates north of 95 percent, ensuring the team is meeting SLAs and is well-positioned for successful disaster recovery;
- Made more efficient use of backup storage with over 90 percent deduplication rates for data backups; and,
- AMP is currently implementing an enterprise data warehouse for plant and process data to have a single source of the truth for monitoring and managing plant operations.
When I started at the organization approximately five years ago, we knew we had a lot of business opportunities coming down the pipeline; for example, we had four hydroelectric plants coming online within 18 months of each other that were all built in pretty much the same way for efficiency’s sake. By applying this concept of uniformity to how we treat data from our critical systems by managing it within a single warehouse and through a single interface, we’re able to use predictive analytics from a fleet versus plant-management perspective. Analytics help us make the most efficient use of budget dollars while ensuring we have an effective plan for command and control in the event of power outages.
For us, the return on investment and return of time invested didn’t take long to achieve.
What we're doing with these systems at a company level is changing the mentality of the way we think. Is the convergence approach cutting edge? Absolutely not. People do this all the time. In the utility industry, it is a little bit of a forward-thinking approach to adapting to the environment. By thinking like a business leader instead of an IT leader, we’re able to provide the highest level of sophistication and security that’s needed to run the facility, but at the most cost effective price.
As AMP brings on more than $2.5 billion in run-of-river hydro generation power plants over the next 18 months, we plan to continue to take advantage of the strategic technologies that are around us, including working with our data management partner to back up, recover and protect data in those facilities and ensure it can be recovered in the event of system failure. In short, we're taking advantage of the opportunities that are around us—and urge you to do the same.
Branndon Kelley is senior vice president of information technology & CIO at American Municipal Power. This article originally appeared on Intelligent Utility and is republished here with permission.
