It’s rare today to find yourself or your organization lacking adequate data in a given situation. Rather, there is plenty of data to support decisions, but often it’s within silos and not rolled up into a more complete picture. If anything, our organizations are adrift in a sea of big data, and we’re searching for analytical islands that can be used to provide some frame of reference from which a reasonable course can be charted. Unfortunately, as pressure increases to make sound decisions, relying solely on these analytical islands is decreasingly defensible.
The challenge with big data is not one of having data, nor is it a matter of knowing how to perform analysis on a given data set. Instead, the challenge is in producing something useful and meaningful that ties together multiple analytical points. That is, rather than relying on siloed analytics in narrow contexts, the difficulty comes in finding ways to pull those disparate analytics together to provide a more complete contextual picture. In essence, this is akin to producing a large-scale composite map from a collection of individual charts. In other words, it’s like cartography — mapmaking — for data.
Context Is Everything
One of the key selling points for moving to a continuous monitoring model is that it decreases risk. However, defending that claim can be difficult at times. Consider the U.S. State Department, which captured headlines starting in 2010 with its Information Assurance program. A SearchSecurity article in April 2010 talked specifically about some of the key statistics surrounding the threat environment and how the State Department was mediating the associated risks: “In a typical week, the department blocks 3.5 million spam emails, intercepts 4,500 viruses and detects more than a million external probes to its network.”
Effectively dealing with this amount of negative traffic led to positive results, due in large part to getting system administrators to apply security patches in a short time frame, which reduced the window of opportunity for attackers to exploit those weaknesses.
“Using this system, overall risk to the department's key unclassified network has been reduced by about 90 percent at both overseas posts and domestic locations,” the article stated.
These data points are very interesting, but they raise a question (more than two years later): How do they fit into the overall context of a risk management program? There is a classic Rumsfeldian problem here — a quote attributed to former President George W. Bush’s Secretary of Defense Donald Rumsfeld: “We don’t know what we don’t know,” which is to say that intercepting viruses, blocking spam and detecting millions of network probes is interesting. But what’s the greater “big data” context? More importantly, how do we take these analytical components and map them to other key data sets, such as traffic to websites, incident response metrics, virus infection cleanups, mop-up efforts from lost mobile devices and mission performance?
From an enterprise risk management perspective, there is a big picture that must be considered. The real challenge with big data is in going from these individual examples of data analytics to a bigger picture that successfully and meaningfully puts those analytics into the larger full-enterprise context. It’s how we map these analytical islands to each other that ultimately provides the support we need for improved decision quality.
Pockets of Data Everywhere
There is no shortage of data, but there is a bigger question to consider: Is decision quality optimal in light of these data sets or is there room for improvement? In fact, one could even go so far as to wonder if data — such as key performance indicators and various other metrics — are being rolled up into a unified view along with other business statistics.
There are several key data sets within IT that can be useful to measure, including:
- uptime and availability;
- mean time between failure (especially for hardware);
- mean time to repair;
- incoming website access statistics (who’s hitting your sites);
- outbound network/Web access statistics (where your people and data go);
- operational security metrics (firewall blocks, viruses detected, scans detected, vulnerability scan and penetration testing results, denied access and failed logins, etc.); and
- security incident metrics (data breaches, physical incidents, lost devices).
Overall, this world of operational metrics — key performance indicators (KPIs) — is well known. However, pulling them together in a meaningful way can still be difficult and elusive. Specifically consider how operational KPIs like uptime and availability may be impacted by security events, which are typically tracked as separate metrics. Then consider the impact of these statistics on larger business functions and processes.